Hi,
I use shorewall-4.5.4 + lsm-0.143 and it does not seem to work as expected...
When all providers are up, everything seems fine.
When one goes down, lsm says "link <provider> down event"... and
it seems
ok but we then experience some problems such as a few unreachable sites,
DNS problems...
If I remove the downed provider from all confs and restart, everything works
again.
Also, when the providers goes back up, lsm seems not to detect it when I use
external IPs or the next hop. He does if I use the LAN IPs but he
won''t
detect a failure past the box.
Could someone check my confs to see if certain parameters would prevent
graceful deactivation of a provider...?
Maybe the accounting rules that use all 3 providers?
About lsm, do I need to manualy add routes for the checked WAN IPs to go
through the respecting devices (as I read on the net) even though the
devices are in the conf?
I am also not sure about the ttl parameter... Should it match the numbers
of hops from the firewall? By example, if the provider has 192.168.2.254
for lan, a.b.c.d for wan et w.x.y.z for the next hop... if I do the lsm
check on w.x.y.z, should I put a ttl=2? and 1 if I check a.b.c.d?
Thx,
JD
----------------------------------------------------------------------
/etc/shorewall/shorewall.conf
----------------------------------------------------------------------
STARTUP_ENABLED=Yes
VERBOSITY=1
BLACKLIST_LOGLEVELLOG_MARTIANS=Yes
LOG_VERBOSITY=2
LOGALLNEWLOGFILE=/var/log/messages
LOGFORMAT="Shorewall:%s:%s:"
LOGTAGONLY=No
LOGLIMITMACLIST_LOG_LEVEL=info
RELATED_LOG_LEVELSFILTER_LOG_LEVEL=info
SMURF_LOG_LEVEL=info
STARTUP_LOG=/var/log/shorewall-init.log
TCP_FLAGS_LOG_LEVEL=info
CONFIG_PATH="${CONFDIR}/shorewall:${SHAREDIR}/shorewall"
GEOIPDIR=/usr/share/xt_geoip/LE
IPTABLESIPIPSETLOCKFILEMODULESDIRPATH="/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/bin:/usr/local/sbin"
PERL=/usr/bin/perl
RESTOREFILE=restore
SHOREWALL_SHELL=/bin/sh
SUBSYSLOCK=/var/lock/subsys/shorewall
TCACCEPT_DEFAULT=none
DROP_DEFAULT=Drop
NFQUEUE_DEFAULT=none
QUEUE_DEFAULT=none
REJECT_DEFAULT=Reject
RCP_COMMAND=''scp ${files} ${root}@${system}:${destination}''
RSH_COMMAND=''ssh ${root}@${system} ${command}''
ACCOUNTING=Yes
ACCOUNTING_TABLE=filter
ADD_IP_ALIASES=Yes
ADD_SNAT_ALIASES=No
ADMINISABSENTMINDED=Yes
AUTO_COMMENT=Yes
AUTOMAKE=No
BLACKLISTNEWONLY=Yes
CLAMPMSS=No
CLEAR_TC=Yes
COMPLETE=No
DELETE_THEN_ADD=Yes
DETECT_DNAT_IPADDRS=No
DISABLE_IPV6=Yes
DONT_LOADDYNAMIC_BLACKLIST=Yes
EXPAND_POLICIES=Yes
EXPORTMODULES=Yes
FASTACCEPT=No
FORWARD_CLEAR_MARKIMPLICIT_CONTINUE=Yes
IPSET_WARNINGS=Yes
IP_FORWARDING=On
KEEP_RT_TABLES=No
LEGACY_FASTSTART=Yes
LOAD_HELPERS_ONLY=No
MACLIST_TABLE=filter
MACLIST_TTLMANGLE_ENABLED=Yes
MAPOLDACTIONS=No
MARK_IN_FORWARD_CHAIN=No
MODULE_SUFFIX=ko
MULTICAST=No
MUTEX_TIMEOUT=60
NULL_ROUTE_RFC1918=No
OPTIMIZE=0
OPTIMIZE_ACCOUNTING=No
REQUIRE_INTERFACE=No
RESTORE_DEFAULT_ROUTE=Yes
RETAIN_ALIASES=No
ROUTE_FILTER=No
SAVE_IPSETS=No
TC_ENABLED=No
TC_EXPERT=No
TC_PRIOMAP="2 3 3 3 2 3 1 1 2 2 2 2 2 2 2 2"
TRACK_PROVIDERS=No
USE_DEFAULT_RT=No
USE_PHYSICAL_NAMES=No
ZONE2ZONE=2
BLACKLIST_DISPOSITION=DROP
MACLIST_DISPOSITION=REJECT
RELATED_DISPOSITION=ACCEPT
SMURF_DISPOSITION=DROP
SFILTER_DISPOSITION=DROP
TCP_FLAGS_DISPOSITION=DROP
TC_BITSPROVIDER_BITSPROVIDER_OFFSETMASK_BITSZONE_BITS=0
IPSECFILE=zones
----------------------------------------------------------------------
/etc/shorewall/interfaces
----------------------------------------------------------------------
sdsl eth1 dhcp,tcpflags,routefilter,nosmurfs,logmartians,optional
free eth2 dhcp,tcpflags,routefilter,nosmurfs,logmartians,optional
#ovh eth3 dhcp,tcpflags,routefilter,nosmurfs,logmartians,optional
loc eth0 tcpflags,nosmurfs,routeback
vpn tun0 tcpflags,nosmurfs
----------------------------------------------------------------------
/etc/shorewall/providers
----------------------------------------------------------------------
sdsl 1 1 main eth1 <IPWAN_SDSL> track,balance eth0
free 2 2 main eth2 192.168.0.254 track,balance eth0
ovh 3 3 main eth3 192.168.2.254 track,balance eth0
----------------------------------------------------------------------
/etc/shorewall/zones
----------------------------------------------------------------------
fw firewall
loc ipv4
sdsl ipv4
vpn ipv4
free ipv4
ovh ipv4
----------------------------------------------------------------------
/etc/shorewall/policy
----------------------------------------------------------------------
loc fw ACCEPT
fw loc ACCEPT
loc sdsl ACCEPT
fw sdsl ACCEPT
loc vpn ACCEPT
vpn loc ACCEPT
loc free ACCEPT
fw free ACCEPT
loc ovh ACCEPT
fw ovh ACCEPT
sdsl all DROP info
free all DROP info
ovh all DROP info
all all REJECT info
----------------------------------------------------------------------
/etc/shorewall/tcinterfaces
----------------------------------------------------------------------
eth0 internal 1000mbit 1000mbit
eth1 external 4mbit 4mbit
tun0 internal 2mbit 1mbit
eth2 external 6mbit 796kbit
eth3 external 6610kbit 796kbit
----------------------------------------------------------------------
/etc/shorewall/rtrules
----------------------------------------------------------------------
192.168.16.0/20 <IP>/23 sdsl 1000
----------------------------------------------------------------------
/etc/shorewall/tunnels
----------------------------------------------------------------------
openvpn:1194 sdsl <IP>
...
----------------------------------------------------------------------
/etc/shorewall/rules
----------------------------------------------------------------------
some rules (only the sdsl)...
----------------------------------------------------------------------
/etc/shorewall/masq
----------------------------------------------------------------------
eth1 192.168.16.0/20 <IPWAN_SDSL>
eth2 192.168.16.0/20 192.168.0.251
eth3 192.168.16.0/20 192.168.2.251
eth0 tun0
----------------------------------------------------------------------
/etc/shorewall/accounting
----------------------------------------------------------------------
many accounting rules on all 3 providers like:
client1_sdsl - eth0:192.168.16.22 eth1 tcp
client1_sdsl - eth1 eth0:192.168.16.22 tcp
client1_sdsl - eth0:192.168.16.22 eth1 udp
client1_sdsl - eth1 eth0:192.168.16.22 udp
COUNT client1_sdsl eth0:192.168.16.22 eth1
COUNT client1_sdsl eth1 eth0:192.168.16.22
client1_free - eth0:192.168.16.22 eth2 tcp
client1_free - eth2 eth0:192.168.16.22 tcp
client1_free - eth0:192.168.16.22 eth2 udp
client1_free - eth2 eth0:192.168.16.22 udp
COUNT client1_free eth0:192.168.16.22 eth2
COUNT client1_free eth2 eth0:192.168.16.22
client1_ovh - eth0:192.168.16.22 eth3 tcp
client1_ovh - eth3 eth0:192.168.16.22 tcp
client1_ovh - eth0:192.168.16.22 eth3 udp
client1_ovh - eth3 eth0:192.168.16.22 udp
COUNT client1_ovh eth0:192.168.16.22 eth3
COUNT client1_ovh eth3 eth0:192.168.16.22
...
----------------------------------------------------------------------
/etc/lsm/lsm.conf
----------------------------------------------------------------------
debug=8
defaults {
name=defaults
checkip=127.0.0.1
eventscript=/etc/lsm/script
notifyscript=/usr/share/lsm/default_script
max_packet_loss=15
max_successive_pkts_lost=7
min_packet_loss=5
min_successive_pkts_rcvd=10
interval_ms=1000
timeout_ms=1000
warn_email=alert@ipernity.com
check_arp=0
sourceip ttl=0
}
include /etc/lsm/shorewall.conf
----------------------------------------------------------------------
/etc/lsm/shorewall.conf
----------------------------------------------------------------------
connection {
name=sdsl
checkip=<EXTERNAL_IP_SDSL>
device=eth1
ttl=2
}
connection {
name=free
checkip=<EXTERNAL_IP_FREE>
device=eth2
ttl=2
}
connection {
name=ovh
checkip=<EXTERNAL_IP_OVH>
device=eth3
ttl=2
}
------------------------------------------------------------------------------
How ServiceNow helps IT people transform IT departments:
1. Consolidate legacy IT systems to a single system of record for IT
2. Standardize and globalize service processes across IT
3. Implement zero-touch automation to replace manual, redundant tasks
http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
On 09/10/2013 08:24 AM, John Doe wrote:> Hi, > > I use shorewall-4.5.4 + lsm-0.143 and it does not seem to work as expected... > When all providers are up, everything seems fine. > > When one goes down, lsm says "link <provider> down event"... and it seems > > ok but we then experience some problems such as a few unreachable sites, > > DNS problems... > If I remove the downed provider from all confs and restart, everything works again.DNS problems can be eliminated by running your own caching name server.> > Also, when the providers goes back up, lsm seems not to detect it when I use > external IPs or the next hop. He does if I use the LAN IPs but he won''t > detect a failure past the box.Your main routing table *must* contain host entries for the external IPs out of the appropriate interface.> > Could someone check my confs to see if certain parameters would prevent > graceful deactivation of a provider...? > Maybe the accounting rules that use all 3 providers? > > About lsm, do I need to manualy add routes for the checked WAN IPs to go > through the respecting devices (as I read on the net) even though the > devices are in the conf?Yes!> I am also not sure about the ttl parameter... Should it match the numbers > of hops from the firewall?Yes. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
On 9/10/2013 11:24 AM, John Doe wrote:> > ---------------------------------------------------------------------- > /etc/lsm/lsm.conf > ---------------------------------------------------------------------- > > debug=8 > defaults { > name=defaults > checkip=127.0.0.1 > eventscript=/etc/lsm/script > notifyscript=/usr/share/lsm/default_script > max_packet_loss=15 > max_successive_pkts_lost=7 > min_packet_loss=5 > min_successive_pkts_rcvd=10 > interval_ms=1000 > timeout_ms=1000 > warn_email=alert@ipernity.com > check_arp=0 > sourceip> ttl=0 > } > include /etc/lsm/shorewall.conf >One pitfall that I found is that "status=1" is set to zero by default in the stock /etc/lsm/lsm.conf file. The Shorewall scripts in the MultiISP document depend on LSM assuming that the link is "up" when LSM starts. Since the stock config is "status=0", LSM is assuming that links are "down" when LSM starts. ------------------------------------------------------------------------------ How ServiceNow helps IT people transform IT departments: 1. Consolidate legacy IT systems to a single system of record for IT 2. Standardize and globalize service processes across IT 3. Implement zero-touch automation to replace manual, redundant tasks http://pubads.g.doubleclick.net/gampad/clk?id=51271111&iu=/4140/ostg.clktrk
From: Tom Eastep <teastep@shorewall.net>> To: shorewall-users@lists.sourceforge.net > Cc: > Sent: Tuesday, September 10, 2013 6:34 PM > Subject: Re: [Shorewall-users] lsm configuration issues... > > On 09/10/2013 08:24 AM, John Doe wrote: >> Hi, >> >> I use shorewall-4.5.4 + lsm-0.143 and it does not seem to work as > expected... >> When all providers are up, everything seems fine. >> >> When one goes down, lsm says "link <provider> down > event"... and it seems >> >> ok but we then experience some problems such as a few unreachable sites, >> >> DNS problems... >> If I remove the downed provider from all confs and restart, everything > works again. > > DNS problems can be eliminated by running your own caching name server. > >> >> Also, when the providers goes back up, lsm seems not to detect it when I > use >> external IPs or the next hop. He does if I use the LAN IPs but he > won''t >> detect a failure past the box. > > Your main routing table *must* contain host entries for the external IPs > out of the appropriate interface. > >> >> Could someone check my confs to see if certain parameters would prevent >> graceful deactivation of a provider...? >> Maybe the accounting rules that use all 3 providers? >> >> About lsm, do I need to manualy add routes for the checked WAN IPs to go >> through the respecting devices (as I read on the net) even though the >> devices are in the conf? > > Yes! > >> I am also not sure about the ttl parameter... Should it match the numbers >> of hops from the firewall? > > Yes.Finally found the time to test and... it still fails... If I put the next hop, with the manual routes (which do work with a ping test), lsm will correctly detect the link down, but will never detect the link back up (even if I have no problem manually pinging the next hop). If I put the external IP of the ADSL box, lsm will of course not see if the link is down past the box. Also, I caught many times zombie lsms that I had to kill manually... Is there a way to at least do lsm jobs manually? If I know one adsl link is down, what can I do to gracefully remove it from shorewall without having to go through all the configuration files to comment references to it? From: Thomas Harold <thomas-lists@nybeta.com>> One pitfall that I found is that "status=1" is set to zero by default > in > the stock /etc/lsm/lsm.conf file.In my version of lsm, it says: # assume initial up state at lsm startup (1 = up, 0 = down, 2 = unknown (default)) So unknown seems to be the default now... Thx, JD ------------------------------------------------------------------------------ LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk
On 9/23/2013 3:05 AM, John Doe wrote:> > Finally found the time to test and... it still fails... > If I put the next hop, with the manual routes (which do work with a ping test), > lsm will correctly detect the link down, but will never detect the link back up (even if > I have no problem manually pinging the next hop). > If I put the external IP of the ADSL box, lsm will of course not see if the link is down > past the box. > Also, I caught many times zombie lsms that I had to kill manually...If you need LSM help, you need to consult the LSM mailing list.> > Is there a way to at least do lsm jobs manually?Of course. LSM just runs /etc/lsm/script. From http://www.shorewall.net/MultiISP.html#LSM if [ ${STATE} = up ]; then ${VARDIR}/firewall enable ${DEVICE} else ${VARDIR}/firewall disable ${DEVICE} fi Rather than running ${VARDIR}/firewall, you can just run ${SBINDIR}/shorewall (usually /sbin/shorewall or /usr/sbin/shorewall): shorewall disable <device-or-provider> shorewall enable <device-provider> -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk
From: Tom Eastep <teastep@shorewall.net>> Rather than running ${VARDIR}/firewall, you can just run > ${SBINDIR}/shorewall (usually /sbin/shorewall or /usr/sbin/shorewall): > > shorewall disable <device-or-provider> > shorewall enable <device-provider>Thx, JD ------------------------------------------------------------------------------ LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151&iu=/4140/ostg.clktrk
On Thu, 12 Sep 2013 03:24:40 -0400 Thomas Harold <thomas-lists@nybeta.com> wrote:> One pitfall that I found is that "status=1" is set to zero by default > in the stock /etc/lsm/lsm.conf file.That''s not true. Default of status is 2 eg unknown.> The Shorewall scripts in the MultiISP document depend on LSM assuming > that the link is "up" when LSM starts. Since the stock config is > "status=0", LSM is assuming that links are "down" when LSM starts.You should really let lsm to control shorewall - not shorewall to control lsm, that''s how lsm is designed to work. That''s pretty simple on redhat based systems where you do something like this: Note, I only use lsm for ipv4 here. /etc/sysconfig/lsm: #!/bin/sh # # LSM to Shorewall Multi-ISP integration script # # Copyright © 2009-2013 Tuomo Soini <tis@foobar.fi> # DAEMON_COREFILE_LIMIT=unlimited VARDIR=$(/sbin/shorewall show vardir) if [ $1 = start -o $1 = restart ]; then # # Set all interfaces to up state on startup. # We will fail soon enough. # rm -f ${VARDIR:-/var/lib/shorewall}/*.status # umask=$(umask) umask 077 >/dev/null /sbin/shorewall -q restart >> /var/log/lsm 2>&1 /sbin/shorewall show routing >> /var/log/lsm umask ${umask} >/dev/null fi That''s how I run lsm with shorewall on rhel6 based system. On connection it''s important to have: eventscript=/usr/share/lsm/shorewall_script -- Tuomo Soini <tis@foobar.fi> Foobar Linux services +358 40 5240030 Foobar Oy <http://foobar.fi/> ------------------------------------------------------------------------------ October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register > http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk