Tyler Walters
2011-Jul-03 05:28 UTC
Multi-ISP from fw only using OpenVPN tun0 as second ISP for one user
Hello,
I have a server with 5 public facing ips, and one OpenVPN tun
connection. The 5 ips are all from the same provider and face the same
gateway. I would eventually like to route all of one user''s traffic to
and from the VPN while leaving the rest of the server''s traffic
untouched. There is no local lan, and the firewall is also the server
-- everything resides on $FW.
I have tried this from a number of angles, so I setup a VMWare machine
to run a limited test before migrating it to the full scale server. I
am testing using "ping -I tun0 google.ca" and "ping
google.ca", where
the first one should route to and from tun0 only, and the second to
and from eth0 only (by default). tun0 will always be assigned the
static ip of 10.88.0.6 and eth0 always 192.168.217.128. The tunnel has
been sucessfully tested and monitored using tshark on both ends of the
tunnel, and on all interfaces (both tun* and eth* at each side). Below
is version information, the commands that successfully work WITHOUT
shorewall being installed at all, and attached is a dump of all config
files as well as a "shorewall dump". Thanks for your help, hopefully
this is easier than I find it to be thus far.
Version information!
root@ubuntu ~ # shorewall version
4.4.20.3
root@ubuntu ~ # uname -a
Linux ubuntu 2.6.38-8-generic #42-Ubuntu SMP Mon Apr 11 03:31:24
UTC 2011 x86_64 x86_64 x86_64 GNU/Linux
root@ubuntu ~ # cat /etc/issue
Ubuntu 11.04
---------------
Now then, the following commands make this work, but I do not know how
to integrate this into shorewall. This is what I suppose my real
question is! I am wrong to assume that this can be carefully put into
shorewall''s config files? As follows...
ip route add 192.168.217.0/24 dev eth0 src 192.168.217.128 table ISP
ip route add default via 192.168.217.2 table ISP
ip route add 10.88.0.0/24 dev tun0 src 10.88.0.6 table VPN
ip route add default via 10.88.0.5 table VPN
ip route add 192.168.217.0/24 dev eth0 src 192.168.217.128
ip route add 10.88.0.0/24 dev tun0 src 10.88.0.6
ip route add default via 192.168.217.2
ip rule add from 192.168.217.128 table ISP
ip rule add from 10.88.0.6 table VPN
In addition, the following has been added to /etc/iproute2/rt_tables,
although ISP is unused...
1 ISP
2 VPN
This yeilds the following...
root@ubuntu ~ # shorewall show routing
Shorewall 4.4.20.3 Routing at ubuntu - Sat Jul 2 21:21:03 PDT 2011
Routing Rules
0: from all lookup local
32764: from 10.88.0.6 lookup VPN
32765: from 192.168.217.128 lookup ISP
32766: from all lookup main
32767: from all lookup default
Table default:
Table ISP:
192.168.217.0/24 dev eth0 scope link src 192.168.217.128
default via 192.168.217.2 dev eth0
Table local:
broadcast 127.255.255.255 dev lo proto kernel scope link src 127.0.0.1
local 192.168.217.128 dev eth0 proto kernel scope host src
192.168.217.128
local 10.88.0.6 dev tun0 proto kernel scope host src 10.88.0.6
broadcast 192.168.217.0 dev eth0 proto kernel scope link src
192.168.217.128
broadcast 192.168.217.255 dev eth0 proto kernel scope link src
192.168.217.128
broadcast 127.0.0.0 dev lo proto kernel scope link src 127.0.0.1
local 127.0.0.1 dev lo proto kernel scope host src 127.0.0.1
local 127.0.0.0/8 dev lo proto kernel scope host src 127.0.0.1
Table main:
10.88.0.1 via 10.88.0.5 dev tun0
10.88.0.5 dev tun0 proto kernel scope link src 10.88.0.6
192.168.217.0/24 dev eth0 proto kernel scope link src 192.168.217.128
10.88.0.0/24 dev tun0 scope link src 10.88.0.6
default via 192.168.217.2 dev eth0
default via 192.168.217.2 dev eth0 metric 100
Table VPN:
10.88.0.0/24 dev tun0 scope link src 10.88.0.6
default via 10.88.0.5 dev tun0
Best Regards,
Tyler
------------------------------------------------------------------------------
All of the data generated in your IT infrastructure is seriously valuable.
Why? It contains a definitive record of application performance, security
threats, fraudulent activity, and more. Splunk takes this data and makes
sense of it. IT sense. And common sense.
http://p.sf.net/sfu/splunk-d2d-c2
Tom Eastep
2011-Jul-03 17:31 UTC
Re: Multi-ISP from fw only using OpenVPN tun0 as second ISP for one user
On Jul 2, 2011, at 10:28 PM, Tyler Walters wrote:> Hello, > > I have a server with 5 public facing ips, and one OpenVPN tun > connection. The 5 ips are all from the same provider and face the same > gateway. I would eventually like to route all of one user''s traffic to > and from the VPN while leaving the rest of the server''s traffic > untouched. There is no local lan, and the firewall is also the server > -- everything resides on $FW. > > I have tried this from a number of angles, so I setup a VMWare machine > to run a limited test before migrating it to the full scale server. I > am testing using "ping -I tun0 google.ca" and "ping google.ca", where > the first one should route to and from tun0 only, and the second to > and from eth0 only (by default). tun0 will always be assigned the > static ip of 10.88.0.6 and eth0 always 192.168.217.128. The tunnel has > been sucessfully tested and monitored using tshark on both ends of the > tunnel, and on all interfaces (both tun* and eth* at each side). Below > is version information, the commands that successfully work WITHOUT > shorewall being installed at all, and attached is a dump of all config > files as well as a "shorewall dump". Thanks for your help, hopefully > this is easier than I find it to be thus far.Don''t use either the route_rules or routes file and simply put this in your /etc/shorewall/providers: #PROVIDER NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY ISP 1 - main eth0 192.168.217.2 track,balance none VPN 2 - main tun0 10.88.0.5 - none That''s it! -Tom Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2
Tom Eastep
2011-Jul-03 17:41 UTC
Re: Multi-ISP from fw only using OpenVPN tun0 as second ISP for one user
On Jul 3, 2011, at 10:31 AM, Tom Eastep wrote:> > On Jul 2, 2011, at 10:28 PM, Tyler Walters wrote: > >> Hello, >> >> I have a server with 5 public facing ips, and one OpenVPN tun >> connection. The 5 ips are all from the same provider and face the same >> gateway. I would eventually like to route all of one user''s traffic to >> and from the VPN while leaving the rest of the server''s traffic >> untouched. There is no local lan, and the firewall is also the server >> -- everything resides on $FW. >> >> I have tried this from a number of angles, so I setup a VMWare machine >> to run a limited test before migrating it to the full scale server. I >> am testing using "ping -I tun0 google.ca" and "ping google.ca", where >> the first one should route to and from tun0 only, and the second to >> and from eth0 only (by default). tun0 will always be assigned the >> static ip of 10.88.0.6 and eth0 always 192.168.217.128. The tunnel has >> been sucessfully tested and monitored using tshark on both ends of the >> tunnel, and on all interfaces (both tun* and eth* at each side). Below >> is version information, the commands that successfully work WITHOUT >> shorewall being installed at all, and attached is a dump of all config >> files as well as a "shorewall dump". Thanks for your help, hopefully >> this is easier than I find it to be thus far. > > > Don''t use either the route_rules or routes file and simply put this in your /etc/shorewall/providers: > > #PROVIDER NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY > ISP 1 - main eth0 192.168.217.2 track,balance none > VPN 2 - main tun0 10.88.0.5 - none >You probably want ''track'' on VPN as well -- sorry for the omission. -Tom Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2
Tyler Walters
2011-Jul-03 17:43 UTC
Re: Multi-ISP from fw only using OpenVPN tun0 as second ISP for one user
Amazing, and here I though I had missed something and needed more config files to make this happen -- thanks a million! It works with the ping test no problem. Now, for the last part. I would like to have all traffic from the user "deluge" to be routed over OpenVPN via tun0, but all other traffic over the ISP via eth0. I have this in my tcrules file: #MARK SOURCE DEST PROTO DEST SOURCE USER TEST LENGTH TOS CONNBYTES HELPER # PORT(S) PORT(S) 0x200:T $FW 0.0.0.0/0 - - - deluge But, it appears that it does not re-route packets as required. Best Regards, Tyler On 3 July 2011 13:31, Tom Eastep <teastep@shorewall.net> wrote:> > On Jul 2, 2011, at 10:28 PM, Tyler Walters wrote: > >> Hello, >> >> I have a server with 5 public facing ips, and one OpenVPN tun >> connection. The 5 ips are all from the same provider and face the same >> gateway. I would eventually like to route all of one user''s traffic to >> and from the VPN while leaving the rest of the server''s traffic >> untouched. There is no local lan, and the firewall is also the server >> -- everything resides on $FW. >> >> I have tried this from a number of angles, so I setup a VMWare machine >> to run a limited test before migrating it to the full scale server. I >> am testing using "ping -I tun0 google.ca" and "ping google.ca", where >> the first one should route to and from tun0 only, and the second to >> and from eth0 only (by default). tun0 will always be assigned the >> static ip of 10.88.0.6 and eth0 always 192.168.217.128. The tunnel has >> been sucessfully tested and monitored using tshark on both ends of the >> tunnel, and on all interfaces (both tun* and eth* at each side). Below >> is version information, the commands that successfully work WITHOUT >> shorewall being installed at all, and attached is a dump of all config >> files as well as a "shorewall dump". Thanks for your help, hopefully >> this is easier than I find it to be thus far. > > > Don''t use either the route_rules or routes file and simply put this in your /etc/shorewall/providers: > > #PROVIDER NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY > ISP 1 - main eth0 192.168.217.2 track,balance none > VPN 2 - main tun0 10.88.0.5 - none > > That''s it! > > -Tom > Tom Eastep \ When I die, I want to go like my Grandfather who > Shoreline, \ died peacefully in his sleep. Not screaming like > Washington, USA \ all of the passengers in his car > http://shorewall.net \________________________________________________ > > > > ------------------------------------------------------------------------------ > All of the data generated in your IT infrastructure is seriously valuable. > Why? It contains a definitive record of application performance, security > threats, fraudulent activity, and more. Splunk takes this data and makes > sense of it. IT sense. And common sense. > http://p.sf.net/sfu/splunk-d2d-c2 > _______________________________________________ > Shorewall-users mailing list > Shorewall-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/shorewall-users > >------------------------------------------------------------------------------ All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2
Tyler Walters
2011-Jul-03 17:48 UTC
Re: Multi-ISP from fw only using OpenVPN tun0 as second ISP for one user
>On Jul 3, 2011, at 10:31 AM, Tom Eastep wrote: > >> >> On Jul 2, 2011, at 10:28 PM, Tyler Walters wrote: >> >>> Hello, >>> >>> I have a server with 5 public facing ips, and one OpenVPN tun >>> connection. The 5 ips are all from the same provider and face the same >>> gateway. I would eventually like to route all of one user''s traffic to >>> and from the VPN while leaving the rest of the server''s traffic >>> untouched. There is no local lan, and the firewall is also the server >>> -- everything resides on $FW. >>> >>> I have tried this from a number of angles, so I setup a VMWare machine >>> to run a limited test before migrating it to the full scale server. I >>> am testing using "ping -I tun0 google.ca" and "ping google.ca", where >>> the first one should route to and from tun0 only, and the second to >>> and from eth0 only (by default). tun0 will always be assigned the >>> static ip of 10.88.0.6 and eth0 always 192.168.217.128. The tunnel has >>> been sucessfully tested and monitored using tshark on both ends of the >>> tunnel, and on all interfaces (both tun* and eth* at each side). Below >>> is version information, the commands that successfully work WITHOUT >>> shorewall being installed at all, and attached is a dump of all config >>> files as well as a "shorewall dump". Thanks for your help, hopefully >>> this is easier than I find it to be thus far. >> >> >> Don''t use either the route_rules or routes file and simply put this in your /etc/shorewall/providers: >> >> #PROVIDER NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY >> ISP 1 - main eth0 192.168.217.2 track,balance none >> VPN 2 - main tun0 10.88.0.5 - none >> > > You probably want ''track'' on VPN as well -- sorry for the omission. > > -Tom > > Tom Eastep \ When I die, I want to go like my Grandfather who > Shoreline, \ died peacefully in his sleep. Not screaming like > Washington, USA \ all of the passengers in his car > http://shorewall.net \________________________________________________ > > > > ------------------------------------------------------------------------------ > All of the data generated in your IT infrastructure is seriously valuable. > Why? It contains a definitive record of application performance, security > threats, fraudulent activity, and more. Splunk takes this data and makes > sense of it. IT sense. And common sense. > http://p.sf.net/sfu/splunk-d2d-c2 > _______________________________________________ > Shorewall-users mailing list > Shorewall-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/shorewall-users > >Thanks, added it now. Also, sorry for the top post.. here is my last email again: ---- Amazing, and here I though I had missed something and needed more config files to make this happen -- thanks a million! It works with the ping test no problem. Now, for the last part. I would like to have all traffic from the user "deluge" to be routed over OpenVPN via tun0, but all other traffic over the ISP via eth0. I have this in my tcrules file: #MARK SOURCE DEST PROTO DEST SOURCE USER TEST LENGTH TOS CONNBYTES HELPER # PORT(S) PORT(S) 0x200:T $FW 0.0.0.0/0 - - - deluge But, it appears that it does not re-route packets as required, everything goes through eth0. ------------------------------------------------------------------------------ All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2
Tom Eastep
2011-Jul-03 19:37 UTC
Re: Multi-ISP from fw only using OpenVPN tun0 as second ISP for one user
On Jul 3, 2011, at 10:48 AM, Tyler Walters wrote:> > Now, for the last part. I would like to have all traffic from the user > "deluge" to be routed over OpenVPN via tun0, but all other traffic > over the ISP via eth0. I have this in my tcrules file: > #MARK SOURCE DEST PROTO DEST SOURCE USER > TEST LENGTH TOS CONNBYTES HELPER > # PORT(S) PORT(S) > 0x200:T $FW 0.0.0.0/0 - - - deluge > > But, it appears that it does not re-route packets as required, > everything goes through eth0. >You are marking the packets in the *POSTROUTING* chain -- how could that possibly affect routing? -Tom Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2d-c2