Hi, I have a multi-isp configuration both on ppp interfaces. As one of them is 32Mbit/s and the other is 8Mbit/s , I have a weight setting of 4 to 1 as in the following providers file entries: vdsl 1 0x10000 - ppp1 - track,balance=4 adsl 2 0x20000 - ppp0 - track,balance=1 I would also like to have fallback between them so that if one is disconnected, all packets can go through the other one. How can I achieve this? Thanks. ------------------------------------------------------------------------------ How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
On 9/11/2013 10:42 AM, İlker Aktuna wrote:> Hi, > > I have a multi-isp configuration both on ppp interfaces. > As one of them is 32Mbit/s and the other is 8Mbit/s , I have a weight setting of 4 to 1 as in the following providers file entries: > > vdsl 1 0x10000 - ppp1 - track,balance=4 > adsl 2 0x20000 - ppp0 - track,balance=1 > > I would also like to have fallback between them so that if one is disconnected, all packets can go through the other one. > How can I achieve this?See http://www1.shorewall.net/MultiISP.html#LinkMonitor. Note however that all existing connections through the failed provider will be terminated and must be restarted through the remaining provider. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
Thanks for the quick response. I've read the information but it is too complicated for me. I didn't understand what configuration I should prepare for my setup. So, I'm looking for an easier way. In fact, I don't need to track any IP addresses as both connections are ppp, The interface comes down when a connection failure occurs. In that case, I should lose the route over that interface and automatically all packets should be routed through the remaining link. Am I wrong ? Btw, as the connections are ppp, I don't have seperate IP addresses to track for each connection. In that case, how can I write the LSM config accordingly ? Thanks. -----Original Message----- From: Tom Eastep [mailto:teastep@shorewall.net] Sent: Wednesday, September 11, 2013 8:48 PM To: shorewall-users@lists.sourceforge.net Subject: Re: [Shorewall-users] Fallback in a multi-isp configuration On 9/11/2013 10:42 AM, İlker Aktuna wrote:> Hi, > > I have a multi-isp configuration both on ppp interfaces. > As one of them is 32Mbit/s and the other is 8Mbit/s , I have a weight setting of 4 to 1 as in the following providers file entries: > > vdsl 1 0x10000 - ppp1 - track,balance=4 > adsl 2 0x20000 - ppp0 - track,balance=1 > > I would also like to have fallback between them so that if one is disconnected, all packets can go through the other one. > How can I achieve this?See http://www1.shorewall.net/MultiISP.html#LinkMonitor. Note however that all existing connections through the failed provider will be terminated and must be restarted through the remaining provider. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
On 9/11/2013 3:36 PM, İlker Aktuna wrote:> Thanks for the quick response. I've read the information but it is too complicated for me. > I didn't understand what configuration I should prepare for my setup. >We have a very similar setup, I'll try and put notes together tomorrow and post them to the mailing list. For us, we have a T1 line (1.5Mbps, static IP) and a cable modem (20-30Mbps in / 2-3Mbps out, dynamic IP). Because our disparity was so large and because there was other traffic across the T1, I set the cable modem provider to have a weight of "100" and the T1 has a weight of only "1". I just put this together about a week ago, so I need to go back and see what configuration files I changed, what scripts I had to write, etc. ------------------------------------------------------------------------------ How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
Note: Our setup is a CentOS 6 box, running Shorewall 4.5.4 (from the
Prereqs:
First off, you should print out the MultiISP document from the Shorewall
website and start marking it up with notes:
http://shorewall.net/MultiISP.html
About 98% of the answers are in the MultiISP file or one of the other
shorewall documents. The other 2% are found on the mailing list.
A) I suggest renaming your ethernet devices to have more meaningful
names. In a CentOS environment, we do this in
/etc/sysconfig/network-scripts by:
- Renaming ifcfg-eth0 to ifcfg-newname
- Editing ifcfg-newname and using DEVICE=newname
We used "wancbl" (our cable modem, 25-30Mbps inbound, 2-3Mbps
outbound,
DHCP address), "want1" (our T1 line, static IP), "lan1" and
"lan2" for
the internal ethernet ports.
This made it much easier for us to see traffic when looking using
"atop"
or other tools, rather then trying to remember eth0..eth3 and which was
what.
B) Turn off the NetworkManager service if you have it installed. Simply
doing "chkconfig NetworkManager off" takes care of it in CentOS/RHEL.
- The number one bugaboo when you have NetworkManager installed is that
it likes to overwrite your /etc/resolv.conf file (and does other things
not suitable for a firewall server).
C) We bonded our two LAN ethernet ports (lan1 and lan2) into a simple
active-backup (mode 1) bond called "bond0". So wherever we refer to
"bond0", that is our internal LAN address.
Setting up bonding is not hard, but beyond the scope of this.
D) We are still using IPv4 for everything. While our T1 provider is
talking about IPv6 rollout soon, our cable modem provider has no plans
at all.
E) Use a version control system for /etc, /usr/local and any other
directories where configuration files might be stored. My
recommendation has been FSVS (stores data into a SVN repository) for a
number of years. But you could also use git, hg, cvs, etc.
- It allows you to easily revert changes, without having to reach for
the backup tapes.
- You can leave notes to yourself in the commit messages (in addition to
leaving comments inside the configuration file). With FSVS, I can look
back at the entire history of the server and see when I installed and
configured a particular bit of software, what files I touched, and what
I changed.
- Doing a diff between the configuration right after you installed
something and after you mucked it up makes it easy to undo changes (or
see how you did something 2-3 years from now).
...
Files that we changed from the Shorewall 4.5.4 defaults:
A) Zones file (self-explanatory, we don''t yet use a DMZ zone or
anything
fancy like VLANs):
/etc/shorewall/zones
fw firewall
loc ipv4
net ipv4
B) Interfaces (options may not be 100% correct, but seems to work):
/etc/shorewall/interfaces
loc bond0 dhcp,logmartians,nosmurfs,required,tcpflags
net wancbl dhcp,logmartians,nosmurfs,optional,tcpflags
net want1 logmartians,nosmurfs,optional,tcpflags
In your case, since both WAN interfaces are DHCP, you''ll need to use
the
dhcp option on both. For us, only "wancbl" interface used a DHCP
address.
Since DHCP is in use on the internal LAN, the bond0 interface also
needed the ''dhcp'' option.
C) Policies (you should customize this based on how secure you want
things, letting the $FW talk to everything is probably not best):
/etc/shorewall/policies
loc net ACCEPT
net all DROP info
$FW loc ACCEPT
$FW net ACCEPT
all all REJECT info
D) Rules (you will be doing very custom things to this based on your
needs, we run SSH on multiple ports, only allow SSH from outside via
port 2222/tcp, plus we have NTP and BIND, plus allowing ICMP):
/etc/shorewall/rules
ACCEPT loc $FW tcp 22 # SSHD port
ACCEPT loc $FW tcp 2222 # 2nd SSHD port
ACCEPT net $FW tcp 2222 # only allow SSH over tcp/2222 from outside
ACCEPT $FW net udp ntp
ACCEPT loc $FW udp ntp
ACCEPT $FW loc udp ntp
REDIRECT loc ntp udp ntp # redirect NTP traffic to firewall NTP server
DNS/ACCEPT $FW net
DNS/ACCEPT loc $FW
DNS/ACCEPT $FW loc
DNS/ACCEPT loc net
ACCEPT loc $FW icmp
ACCEPT $FW loc icmp
ACCEPT $FW net icmp
ACCEPT net $FW icmp
E) Masq file (NAT for the LAN clients to the internet). The example
here uses 192.168.0.0/24 as our internal LAN address range (in reality
we use a 172.x.y.z range). The important thing is that you need to list
both WAN ethernet devices, with the same internal LAN address range for
both lines.:
/etc/shorewall/masq
wancbl 192.168.0.0/24
want1 192.168.0.0/24
F) Providers (I am still a bit fuzzy about things here). Note that we
use TRACK_PROVIDERS=Yes and USE_DEFAULT_RT=Yes in the shorewall.conf
file. This affects how you define your two providers:
/etc/shorewall/providers
provcbl 1 2 - wancbl detect balance=100
provt1 2 2 - want1 999.999.999.999 balance=1
- Replace 999.999.999.999 with "detect" if you are using DHCP, or
enter
the static external IP address assigned to that interface.
- USE_DEFAULT_RT=Yes means that the DUPLICATE column should be left as
''-'' in the providers file.
- TRACK_PROVIDERS=Yes automatically adds "track" to the OPTIONS
column,
which is why I don''t specify it as "track,balance=###".
- The 100:1 weight means that 99% of all traffic goes out over the
"wancbl" interface. Only if the cable modem is down, does it start to
route traffic over our T1 line.
- The provider names (1st column) will also show up in your routing
tables (see "shorewall show routing"). So we prefix our provider
names
with "prov" to make it clearer.
- If you need specific traffic to go out over the secondary provider,
then you need to play with "rtrules" or "tcrules" (we
don''t use this
yet, so that is not a certainty). Get it working first without touching
those files.
- I have been unable to get "fallback" option working properly in
Shorewall 4.5.4. But the above works well enough for our purposes.
G) shorewall.conf (the main config file). I am only listing lines that
changed from the base install. You should *definitely* read the main
Shorewall documentation and make notes in the configuration file for why
you changed a particular value.
/etc/shorewall/shorewall.conf
VERBOSITY=2
NULL_ROUTE_RFC1918=Yes
RESTORE_DEFAULT_ROUTE=No
TRACK_PROVIDERS=Yes
USE_DEFAULT_RT=Yes
H) The "routestopped" file (for Shorewall v4.5.4), it has a different
name in later versions of Shorewall (stoppedrules). These are the
iptables rules that get generate whenever Shorewall shuts down.
bond0 - - icmp
bond0 - - tcp 22
bond0 - - udp 123
bond0 - - tcp 2222
want1 - source icmp
want1 - source tcp 2222
wancbl - source icmp
wancbl - source tcp 2222
...
That''s the end of the *simple* stuff. Other then the providers file
and
defining multiple WAN interfaces, it looks very similar to a single-ISP
setup.
...
LSM configuration notes:
- We used lsm-0.163-1.foo6.src.rpm for our LSM install on CentOS 6.
- The default LSM configuration is not quite correct for working with
Shorewall.
...
A) You need to change a few defaults in the lsm.conf file. These lines
were all added or changed inside the "defaults{}" block.
eventscript=/etc/lsm/script
notifyscriptstatus=1
B) Before the "#EOF" line at the end of the /etc/lsm/lsm.conf file,
you
need to add the following line:
include /var/lib/shorewall/lsm.conf
C) A sample /var/lib/shorewall/lsm.conf
connection {
name=T1
checkip=999.999.999.999
device=want1
ttl=2
}
connection {
name=Cablevision
checkip=999.999.999.999
device=wancbl
ttl=2
}
- You should replace "999.999.999.999" with the current DHCP IP
address
- This file actually gets written out by Shorewall, but for testing LSM,
you will want to initially create it by hand.
D) You do *not* need to run LSM as a service. On CentOS 6, if
"chkconfig --list lsm" shows that LSM is running in runlevels 3-5,
then
you should "chkconfig lsm off" to disable the service.
E) /etc/lsm/script - This is the script that LSM will run whenever the
status of a link changes (eventscript= argument in /etc/lsm/lsm.conf).
See attachment to this email for a copy of our LSM script (which is
slightly different then the Shorewall MultiISP example script).
It needs to perform the following tasks when a link goes "up":
- Remove the [interface].status file
- Execute the Shorewall "firewall enable {device}" script
When the link goes down, it needs to:
- Execute the Shorewall "firewall disable {device}" script
Note: The above only works for Shorewall 4.5.4 and later.
I have added some other logging statements to the script that shoves the
details and events into /var/log/lsm.log.
...
Now that LSM is setup, we can go back to setting up Shorewall to
interact with LSM.
...
A) /etc/shorewall/isusable
This *should* be already setup by default. It looks for
"[interface].status" files in the VARDIR, where LSM will write those
status files.
B) /etc/shorewall/params
You *could* define various things here instead of hardcoding them in the
other files. But we did not do so in our setup.
C) /etc/shorewall/started
This is the shell script commands that get run whenever shorewall starts
up. We decided to use the single solitary line in our "started" file
rather then check whether this was a "restart" command. The
"start_lsm"
is a function defined in /etc/shorewall/lib.private
# [Re]Start LSM
start_lsm
D) /etc/shorewall/restored
This is covered in the multi-ISP document. It checks to see if LSM is
already running, otherwise it starts it.
# Start lsm if it isn''t running
if [ -z "$(ps ax | grep ''lsm '' | grep -v ''grep
'' )" ]; then
start_lsm
fi
E) /etc/shorewall/lib.private
The meat-and-potatoes for interacting with LSM. It defines the
"start_lsm()" shell function. See attached file.
- The checkip line can be written two ways. You can specify a static IP
address that is the "next hop out" for that particular interface. Or
you can use the "SW_(interfacename)_GATEWAY" environment variable for
DHCP setups and also specify a fallback address.
Static version:
checkip=999.999.999.999
DHCP version:
checkip=${SW_WANCBL_GATEWAY:-999.999.999.999}
- 999.999.999.999 in this file needs to be your "next-hop" gateway IP
address for that WAN interface, not the external IP address of your
firewall.
- If you need to go two+ hops out, then you will *also* need to add a
static "ip route". I''m not showing an example of that, and
for cable
modem setups it is probably not needed.
- Our lib.private is somewhat custom, with some additional logging added
that is not in the MultiISP example document.
- start_lsm() is responsible for writing out the /var/shorewall/lsm.conf
file.
- We should be using more of Shorewall variables (such as VARDIR)
instead of hardcoding /var/lib/shorewall in places.
F) /etc/shorewall/findgw
This file may not exist by default, or may not be compatible with your
distro. Sample versions of the "findgw" script can be found at:
http://www.shorewall.net/pub/shorewall/contrib/findgw/
Without the proper "findgw" file, you will see that even though the
interface is up, there is no default route being added to that
provider''s routing table (see "shorewall show routing").
...
I think that''s everything that we changed when setting up our MultiISP
configuration. There are no warranties, but this should serve as a
trail of breadcrumbs to help you figure out what needs to be touched.
------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. Consolidate legacy IT systems to a single system of record for IT
2. Standardize and globalize service processes across IT
3. Implement zero-touch automation to replace manual, redundant tasks
http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
Hi Thomas,
Thanks for this great detailed information.
Unfortunately it is still not very clear for me whast to write instead of your
999.999.999.999 example.
My wan interfaces are ppp0 and ppp1 . They have dynamic IP addresses and their
gateways are same because they connect to the same ISP.
What should I write for them ?
Also, I didn't still install lsm yet but checked fpor availability of the
explained files on my system.
There are no "[interface].status" files under vARDIR
(/var/lib/shorewall)
How will they be populated?
Thanks.
-----Original Message-----
From: Thomas Harold [mailto:thomas-lists@nybeta.com]
Sent: Thursday, September 12, 2013 10:22 AM
To: Shorewall Users
Cc: İlker Aktuna
Subject: Re: [Shorewall-users] Fallback in a multi-isp configuration
Note: Our setup is a CentOS 6 box, running Shorewall 4.5.4 (from the
Prereqs:
First off, you should print out the MultiISP document from the Shorewall website
and start marking it up with notes:
http://shorewall.net/MultiISP.html
About 98% of the answers are in the MultiISP file or one of the other shorewall
documents. The other 2% are found on the mailing list.
A) I suggest renaming your ethernet devices to have more meaningful names. In a
CentOS environment, we do this in /etc/sysconfig/network-scripts by:
- Renaming ifcfg-eth0 to ifcfg-newname
- Editing ifcfg-newname and using DEVICE=newname
We used "wancbl" (our cable modem, 25-30Mbps inbound, 2-3Mbps
outbound, DHCP address), "want1" (our T1 line, static IP),
"lan1" and "lan2" for the internal ethernet ports.
This made it much easier for us to see traffic when looking using
"atop"
or other tools, rather then trying to remember eth0..eth3 and which was what.
B) Turn off the NetworkManager service if you have it installed. Simply doing
"chkconfig NetworkManager off" takes care of it in CentOS/RHEL.
- The number one bugaboo when you have NetworkManager installed is that it likes
to overwrite your /etc/resolv.conf file (and does other things not suitable for
a firewall server).
C) We bonded our two LAN ethernet ports (lan1 and lan2) into a simple
active-backup (mode 1) bond called "bond0". So wherever we refer to
"bond0", that is our internal LAN address.
Setting up bonding is not hard, but beyond the scope of this.
D) We are still using IPv4 for everything. While our T1 provider is talking
about IPv6 rollout soon, our cable modem provider has no plans at all.
E) Use a version control system for /etc, /usr/local and any other directories
where configuration files might be stored. My recommendation has been FSVS
(stores data into a SVN repository) for a number of years. But you could also
use git, hg, cvs, etc.
- It allows you to easily revert changes, without having to reach for the backup
tapes.
- You can leave notes to yourself in the commit messages (in addition to leaving
comments inside the configuration file). With FSVS, I can look back at the
entire history of the server and see when I installed and configured a
particular bit of software, what files I touched, and what I changed.
- Doing a diff between the configuration right after you installed something and
after you mucked it up makes it easy to undo changes (or see how you did
something 2-3 years from now).
...
Files that we changed from the Shorewall 4.5.4 defaults:
A) Zones file (self-explanatory, we don't yet use a DMZ zone or anything
fancy like VLANs):
/etc/shorewall/zones
fw firewall
loc ipv4
net ipv4
B) Interfaces (options may not be 100% correct, but seems to work):
/etc/shorewall/interfaces
loc bond0 dhcp,logmartians,nosmurfs,required,tcpflags
net wancbl dhcp,logmartians,nosmurfs,optional,tcpflags
net want1 logmartians,nosmurfs,optional,tcpflags
In your case, since both WAN interfaces are DHCP, you'll need to use the
dhcp option on both. For us, only "wancbl" interface used a DHCP
address.
Since DHCP is in use on the internal LAN, the bond0 interface also needed the
'dhcp' option.
C) Policies (you should customize this based on how secure you want things,
letting the $FW talk to everything is probably not best):
/etc/shorewall/policies
loc net ACCEPT
net all DROP info
$FW loc ACCEPT
$FW net ACCEPT
all all REJECT info
D) Rules (you will be doing very custom things to this based on your needs, we
run SSH on multiple ports, only allow SSH from outside via port 2222/tcp, plus
we have NTP and BIND, plus allowing ICMP):
/etc/shorewall/rules
ACCEPT loc $FW tcp 22 # SSHD port
ACCEPT loc $FW tcp 2222 # 2nd SSHD port
ACCEPT net $FW tcp 2222 # only allow SSH over tcp/2222 from outside
ACCEPT $FW net udp ntp
ACCEPT loc $FW udp ntp
ACCEPT $FW loc udp ntp
REDIRECT loc ntp udp ntp # redirect NTP traffic to firewall NTP server
DNS/ACCEPT $FW net
DNS/ACCEPT loc $FW
DNS/ACCEPT $FW loc
DNS/ACCEPT loc net
ACCEPT loc $FW icmp
ACCEPT $FW loc icmp
ACCEPT $FW net icmp
ACCEPT net $FW icmp
E) Masq file (NAT for the LAN clients to the internet). The example here uses
192.168.0.0/24 as our internal LAN address range (in reality we use a 172.x.y.z
range). The important thing is that you need to list both WAN ethernet devices,
with the same internal LAN address range for both lines.:
/etc/shorewall/masq
wancbl 192.168.0.0/24
want1 192.168.0.0/24
F) Providers (I am still a bit fuzzy about things here). Note that we use
TRACK_PROVIDERS=Yes and USE_DEFAULT_RT=Yes in the shorewall.conf file. This
affects how you define your two providers:
/etc/shorewall/providers
provcbl 1 2 - wancbl detect balance=100
provt1 2 2 - want1 999.999.999.999 balance=1
- Replace 999.999.999.999 with "detect" if you are using DHCP, or
enter the static external IP address assigned to that interface.
- USE_DEFAULT_RT=Yes means that the DUPLICATE column should be left as
'-' in the providers file.
- TRACK_PROVIDERS=Yes automatically adds "track" to the OPTIONS
column, which is why I don't specify it as "track,balance=###".
- The 100:1 weight means that 99% of all traffic goes out over the
"wancbl" interface. Only if the cable modem is down, does it start to
route traffic over our T1 line.
- The provider names (1st column) will also show up in your routing tables (see
"shorewall show routing"). So we prefix our provider names with
"prov" to make it clearer.
- If you need specific traffic to go out over the secondary provider, then you
need to play with "rtrules" or "tcrules" (we don't use
this yet, so that is not a certainty). Get it working first without touching
those files.
- I have been unable to get "fallback" option working properly in
Shorewall 4.5.4. But the above works well enough for our purposes.
G) shorewall.conf (the main config file). I am only listing lines that changed
from the base install. You should *definitely* read the main Shorewall
documentation and make notes in the configuration file for why you changed a
particular value.
/etc/shorewall/shorewall.conf
VERBOSITY=2
NULL_ROUTE_RFC1918=Yes
RESTORE_DEFAULT_ROUTE=No
TRACK_PROVIDERS=Yes
USE_DEFAULT_RT=Yes
H) The "routestopped" file (for Shorewall v4.5.4), it has a different
name in later versions of Shorewall (stoppedrules). These are the iptables
rules that get generate whenever Shorewall shuts down.
bond0 - - icmp
bond0 - - tcp 22
bond0 - - udp 123
bond0 - - tcp 2222
want1 - source icmp
want1 - source tcp 2222
wancbl - source icmp
wancbl - source tcp 2222
...
That's the end of the *simple* stuff. Other then the providers file and
defining multiple WAN interfaces, it looks very similar to a single-ISP setup.
...
LSM configuration notes:
- We used lsm-0.163-1.foo6.src.rpm for our LSM install on CentOS 6.
- The default LSM configuration is not quite correct for working with Shorewall.
...
A) You need to change a few defaults in the lsm.conf file. These lines were all
added or changed inside the "defaults{}" block.
eventscript=/etc/lsm/script
notifyscriptstatus=1
B) Before the "#EOF" line at the end of the /etc/lsm/lsm.conf file,
you need to add the following line:
include /var/lib/shorewall/lsm.conf
C) A sample /var/lib/shorewall/lsm.conf
connection {
name=T1
checkip=999.999.999.999
device=want1
ttl=2
}
connection {
name=Cablevision
checkip=999.999.999.999
device=wancbl
ttl=2
}
- You should replace "999.999.999.999" with the current DHCP IP
address
- This file actually gets written out by Shorewall, but for testing LSM, you
will want to initially create it by hand.
D) You do *not* need to run LSM as a service. On CentOS 6, if "chkconfig
--list lsm" shows that LSM is running in runlevels 3-5, then you should
"chkconfig lsm off" to disable the service.
E) /etc/lsm/script - This is the script that LSM will run whenever the status of
a link changes (eventscript= argument in /etc/lsm/lsm.conf).
See attachment to this email for a copy of our LSM script (which is slightly
different then the Shorewall MultiISP example script).
It needs to perform the following tasks when a link goes "up":
- Remove the [interface].status file
- Execute the Shorewall "firewall enable {device}" script
When the link goes down, it needs to:
- Execute the Shorewall "firewall disable {device}" script
Note: The above only works for Shorewall 4.5.4 and later.
I have added some other logging statements to the script that shoves the details
and events into /var/log/lsm.log.
...
Now that LSM is setup, we can go back to setting up Shorewall to interact with
LSM.
...
A) /etc/shorewall/isusable
This *should* be already setup by default. It looks for
"[interface].status" files in the VARDIR, where LSM will write those
status files.
B) /etc/shorewall/params
You *could* define various things here instead of hardcoding them in the other
files. But we did not do so in our setup.
C) /etc/shorewall/started
This is the shell script commands that get run whenever shorewall starts up. We
decided to use the single solitary line in our "started" file rather
then check whether this was a "restart" command. The
"start_lsm"
is a function defined in /etc/shorewall/lib.private
# [Re]Start LSM
start_lsm
D) /etc/shorewall/restored
This is covered in the multi-ISP document. It checks to see if LSM is already
running, otherwise it starts it.
# Start lsm if it isn't running
if [ -z "$(ps ax | grep 'lsm ' | grep -v 'grep ' )" ];
then
start_lsm
fi
E) /etc/shorewall/lib.private
The meat-and-potatoes for interacting with LSM. It defines the
"start_lsm()" shell function. See attached file.
- The checkip line can be written two ways. You can specify a static IP address
that is the "next hop out" for that particular interface. Or you can
use the "SW_(interfacename)_GATEWAY" environment variable for DHCP
setups and also specify a fallback address.
Static version:
checkip=999.999.999.999
DHCP version:
checkip=${SW_WANCBL_GATEWAY:-999.999.999.999}
- 999.999.999.999 in this file needs to be your "next-hop" gateway IP
address for that WAN interface, not the external IP address of your firewall.
- If you need to go two+ hops out, then you will *also* need to add a static
"ip route". I'm not showing an example of that, and for cable
modem setups it is probably not needed.
- Our lib.private is somewhat custom, with some additional logging added that is
not in the MultiISP example document.
- start_lsm() is responsible for writing out the /var/shorewall/lsm.conf file.
- We should be using more of Shorewall variables (such as VARDIR) instead of
hardcoding /var/lib/shorewall in places.
F) /etc/shorewall/findgw
This file may not exist by default, or may not be compatible with your distro.
Sample versions of the "findgw" script can be found at:
http://www.shorewall.net/pub/shorewall/contrib/findgw/
Without the proper "findgw" file, you will see that even though the
interface is up, there is no default route being added to that provider's
routing table (see "shorewall show routing").
...
I think that's everything that we changed when setting up our MultiISP
configuration. There are no warranties, but this should serve as a trail of
breadcrumbs to help you figure out what needs to be touched.
------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. Consolidate legacy IT systems to a single system of record for IT
2. Standardize and globalize service processes across IT
3. Implement zero-touch automation to replace manual, redundant tasks
http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
_______________________________________________
Shorewall-users mailing list
Shorewall-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/shorewall-users
On 9/12/2013 3:25 PM, İlker Aktuna wrote:> Hi Thomas, > > Thanks for this great detailed information. Unfortunately it is still > not very clear for me what to write instead of your 999.999.999.999 > example. My wan interfaces are ppp0 and ppp1 . They have dynamic IP > addresses and their gateways are same because they connect to the > same ISP. >Since you are using DHCP on both ppp0 and ppp1, just use "detect" in /etc/shorewall/providers for the GATEWAY column. Or possibly a "-" instead since they are PPP. See point #6 at the following URL. http://shorewall.net/MultiISP.html#USE_DEFAULT_RT Because both of your interfaces talk to the same ISP and have the same "next-hop" or "default" gateway, you should also read the following section and probably use "load=" instead of "balance=" in the providers file. http://shorewall.net/MultiISP.html#load> Also, I didn't still install lsm yet but checked fpor availability of > the explained files on my system. There are no "[interface].status" > files under vARDIR (/var/lib/shorewall) How will they be populated? >LSM creates those /var/lib/shorewall/interfacename.status files. But only when the interface is "down". So until you setup LSM, you won't see .status files show in in /var/lib/shorewall. With your interfaces being named 'ppp0' and 'ppp1' you would see: /var/lib/shorewall/ppp0.status /var/lib/shorewall/ppp1.status To test whether LSM is configured properly and integrating properly with Shorewall, you can "ifdown ppp0" and see whether LSM creates the ppp0.status file (with content of "1" inside). If the .status files are *not* present, then the "/etc/shorewall/isusable" script will decide that the interface is up and running and Shorewall will use it. ... Note that it takes 20-40 seconds for LSM to notice that an interface is down, decide that it is down for good and then restart Shorewall by executing the "firewall disable [interface}" command. When the interface comes back up, it will take 2-3 minutes for LSM to decide that things are okay enough to execute "firewall enable [interface]". So testing this requires a bit of patience. Or you could adjust the LSM defaults in /etc/lsm/lsm.conf to make it decide things faster. Specific attributes that control the decision process and time-to-decide are "max_packet_loss, max_successive_pkts_lost, min_packet_loss, min_successive_pkts_rcvd, interval_ms, timeout_ms". ------------------------------------------------------------------------------ How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
Thanks. I will try that. -----Original Message----- From: Thomas Harold [mailto:thomas-lists@nybeta.com] Sent: Thursday, September 12, 2013 11:12 PM To: İlker Aktuna Cc: 'Shorewall Users' Subject: Re: [Shorewall-users] Fallback in a multi-isp configuration On 9/12/2013 3:25 PM, İlker Aktuna wrote:> Hi Thomas, > > Thanks for this great detailed information. Unfortunately it is still > not very clear for me what to write instead of your 999.999.999.999 > example. My wan interfaces are ppp0 and ppp1 . They have dynamic IP > addresses and their gateways are same because they connect to the same > ISP. >Since you are using DHCP on both ppp0 and ppp1, just use "detect" in /etc/shorewall/providers for the GATEWAY column. Or possibly a "-" instead since they are PPP. See point #6 at the following URL. http://shorewall.net/MultiISP.html#USE_DEFAULT_RT Because both of your interfaces talk to the same ISP and have the same "next-hop" or "default" gateway, you should also read the following section and probably use "load=" instead of "balance=" in the providers file. http://shorewall.net/MultiISP.html#load> Also, I didn't still install lsm yet but checked fpor availability of > the explained files on my system. There are no "[interface].status" > files under vARDIR (/var/lib/shorewall) How will they be populated? >LSM creates those /var/lib/shorewall/interfacename.status files. But only when the interface is "down". So until you setup LSM, you won't see .status files show in in /var/lib/shorewall. With your interfaces being named 'ppp0' and 'ppp1' you would see: /var/lib/shorewall/ppp0.status /var/lib/shorewall/ppp1.status To test whether LSM is configured properly and integrating properly with Shorewall, you can "ifdown ppp0" and see whether LSM creates the ppp0.status file (with content of "1" inside). If the .status files are *not* present, then the "/etc/shorewall/isusable" script will decide that the interface is up and running and Shorewall will use it. ... Note that it takes 20-40 seconds for LSM to notice that an interface is down, decide that it is down for good and then restart Shorewall by executing the "firewall disable [interface}" command. When the interface comes back up, it will take 2-3 minutes for LSM to decide that things are okay enough to execute "firewall enable [interface]". So testing this requires a bit of patience. Or you could adjust the LSM defaults in /etc/lsm/lsm.conf to make it decide things faster. Specific attributes that control the decision process and time-to-decide are "max_packet_loss, max_successive_pkts_lost, min_packet_loss, min_successive_pkts_rcvd, interval_ms, timeout_ms". ------------------------------------------------------------------------------ How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
On 9/12/2013 1:12 PM, Thomas Harold wrote:> On 9/12/2013 3:25 PM, İlker Aktuna wrote: >> Hi Thomas, >> >> Thanks for this great detailed information. Unfortunately it is still >> not very clear for me what to write instead of your 999.999.999.999 >> example. My wan interfaces are ppp0 and ppp1 . They have dynamic IP >> addresses and their gateways are same because they connect to the >> same ISP. >> > > Since you are using DHCP on both ppp0 and ppp1, just use "detect" in > /etc/shorewall/providers for the GATEWAY column. Or possibly a "-" > instead since they are PPP. See point #6 at the following URL. > > http://shorewall.net/MultiISP.html#USE_DEFAULT_RT > > Because both of your interfaces talk to the same ISP and have the same > "next-hop" or "default" gateway, you should also read the following > section and probably use "load=" instead of "balance=" in the providers > file. > > http://shorewall.net/MultiISP.html#load > >> Also, I didn''t still install lsm yet but checked fpor availability of >> the explained files on my system. There are no "[interface].status" >> files under vARDIR (/var/lib/shorewall) How will they be populated? >> > > LSM creates those /var/lib/shorewall/interfacename.status files. But > only when the interface is "down". So until you setup LSM, you won''t > see .status files show in in /var/lib/shorewall. > > With your interfaces being named ''ppp0'' and ''ppp1'' you would see: > > /var/lib/shorewall/ppp0.status > /var/lib/shorewall/ppp1.status > > To test whether LSM is configured properly and integrating properly with > Shorewall, you can "ifdown ppp0" and see whether LSM creates the > ppp0.status file (with content of "1" inside). > > If the .status files are *not* present, then the > "/etc/shorewall/isusable" script will decide that the interface is up > and running and Shorewall will use it.Later versions of Shorewall will create those files if the provider interfaces are ''optional''. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk