I have a dual ISP setup and while I prefer one provider for output by default, I
have come across a situation where I want to force the traffic from a given IP
address on my lan through the (non-default) provider, which I did with a
route_rules entry:
10.75.22.101 - IGS 1002
and that is having the desired effect with a route rule table looking like:
0: from all lookup local
1000: from all to 10.75.23.0/24 lookup main
1001: from all to 10.8.0.0/24 lookup main
1002: from 10.75.22.101 lookup IGS
10000: from all fwmark 0x100 lookup CGCO
10001: from all fwmark 0x200 lookup IGS
20000: from 7.1.7.2 lookup CGCO
20256: from 6.1.3.4 lookup IGS
32766: from all lookup main
32767: from all lookup default
and the particular traffic does seem to be using the correct provider''s
output
interface (ppp0), however the source of the these particularly directed packets
on the ppp0 interface (IGS) is the address of the "preferred"
provider''s
(CGCO)''s interface: 7.1.7.2. I would have thought the following nat
table rules
would have corrected that:
Chain POSTROUTING (policy ACCEPT 782 packets, 47801 bytes)
pkts bytes target prot opt in out source destination
1 1400 ppp0_masq all -- * ppp0 0.0.0.0/0 0.0.0.0/0
2581 201K eth0.1_masq all -- * eth0.1 0.0.0.0/0 0.0.0.0/0
Chain ppp0_masq (1 references)
pkts bytes target prot opt in out source destination
1 1400 SNAT all -- * * !6.1.3.4 0.0.0.0/0 to:6.1.3.4
But according to the tcpdumping on ppp0, it''s not having any effect.
So what am I misunderstanding about all of this?
------------------------------------------------------------------------------
SOLARIS 10 is the OS for Data Centers - provides features such as DTrace,
Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW
http://p.sf.net/sfu/solaris-dev2dev
Brian J. Murrell wrote:> I have a dual ISP setup and while I prefer one provider for output by default, I > have come across a situation where I want to force the traffic from a given IP > address on my lan through the (non-default) provider, which I did with a > route_rules entry: > > 10.75.22.101 - IGS 1002 > > and that is having the desired effect with a route rule table looking like: > > 0: from all lookup local > 1000: from all to 10.75.23.0/24 lookup main > 1001: from all to 10.8.0.0/24 lookup main > 1002: from 10.75.22.101 lookup IGS > 10000: from all fwmark 0x100 lookup CGCO > 10001: from all fwmark 0x200 lookup IGS > 20000: from 7.1.7.2 lookup CGCO > 20256: from 6.1.3.4 lookup IGS > 32766: from all lookup main > 32767: from all lookup default > > and the particular traffic does seem to be using the correct provider''s output > interface (ppp0), however the source of the these particularly directed packets > on the ppp0 interface (IGS) is the address of the "preferred" provider''s > (CGCO)''s interface: 7.1.7.2. I would have thought the following nat table rules > would have corrected that: > > Chain POSTROUTING (policy ACCEPT 782 packets, 47801 bytes) > pkts bytes target prot opt in out source destination > 1 1400 ppp0_masq all -- * ppp0 0.0.0.0/0 0.0.0.0/0 > 2581 201K eth0.1_masq all -- * eth0.1 0.0.0.0/0 0.0.0.0/0 > > Chain ppp0_masq (1 references) > pkts bytes target prot opt in out source destination > 1 1400 SNAT all -- * * !6.1.3.4 0.0.0.0/0 to:6.1.3.4 > > But according to the tcpdumping on ppp0, it''s not having any effect.If it were not having any effect, the SOURCE IP would be 10.75.22.101, right?> > So what am I misunderstanding about all of this? >It''s not clear to me what''s happening from what you have written. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ SOLARIS 10 is the OS for Data Centers - provides features such as DTrace, Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW http://p.sf.net/sfu/solaris-dev2dev
Tom Eastep <teastep <at> shorewall.net> writes:> > If it were not having any effect, the SOURCE IP would be 10.75.22.101, > right?Ahhh. Yes, of course, you are right. So some SNAT/Masq is happening, just for the wrong outgoing interface.> It''s not clear to me what''s happening from what you have written.Sorry. It really is simply a case of using a route_rules entry to force traffic from a given IP (10.75.22.101) out through what is normally not the "default"/preferred interface (which is provider CGCO), which is working. But that the source address of what is normally the preferred/default interface (CGCO''s 7.1.7.2) is being masqued to the packets instead of the source address of the interface (IGS0 being forced by the route_rules entry. What I don''t understand is why given the following nat table rules: Chain POSTROUTING (policy ACCEPT 782 packets, 47801 bytes) pkts bytes target prot opt in out source destination 1 1400 ppp0_masq all -- * ppp0 0.0.0.0/0 0.0.0.0/0 2581 201K eth0.1_masq all -- * eth0.1 0.0.0.0/0 0.0.0.0/0 Chain ppp0_masq (1 references) pkts bytes target prot opt in out source destination 1 1400 SNAT all -- * * !6.1.3.4 0.0.0.0/0 to:6.1.3.4 Chain eth0.1_masq (1 references) pkts bytes target prot opt in out source destination 5344 414K MASQUERADE all -- * * 0.0.0.0/0 0.0.0.0/0 is the eth0.1 masqing being applied to packets which are routed to the IGS (on ppp0) provider with the route rule: 1002: from 10.75.22.101 lookup IGS and associated routing table: Table IGS: 10.8.0.2 dev tun0 proto kernel scope link src 10.8.0.1 192.168.200.1 dev ppp0 scope link src 66.11.173.224 10.10.0.0/24 via 10.75.22.1 dev br-lan proto zebra metric 20 equalize 10.8.0.0/24 via 10.8.0.2 dev tun0 192.168.0.0/24 via 10.75.22.5 dev br-lan proto zebra metric 20 equalize 10.75.22.0/24 dev br-lan proto kernel scope link src 10.75.22.254 10.75.23.0/24 via 10.8.0.2 dev tun0 192.168.122.0/24 via 10.75.22.151 dev br-lan proto zebra metric 20 equalize 169.254.0.0/16 via 10.75.22.5 dev br-lan proto zebra metric 20 equalize default via 192.168.200.1 dev ppp0 src 6.1.3.4 Doesn''t the above set the outgoing interface to ppp0 before POSTROUTING is applied in the iptables "nat" table? ------------------------------------------------------------------------------ SOLARIS 10 is the OS for Data Centers - provides features such as DTrace, Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW http://p.sf.net/sfu/solaris-dev2dev
Brian J. Murrell wrote:> Tom Eastep <teastep <at> shorewall.net> writes: >> If it were not having any effect, the SOURCE IP would be 10.75.22.101, >> right? > > Ahhh. Yes, of course, you are right. So some SNAT/Masq is happening, just for > the wrong outgoing interface. > >> It''s not clear to me what''s happening from what you have written. > > Sorry. It really is simply a case of using a route_rules entry to force > traffic from a given IP (10.75.22.101) out through what is normally not the > "default"/preferred interface (which is provider CGCO), which is working. But > that the source address of what is normally the preferred/default interface > (CGCO''s 7.1.7.2) is being masqued to the packets instead of the source address > of the interface (IGS0 being forced by the route_rules entry. > > What I don''t understand is why given the following nat table rules: > > Chain POSTROUTING (policy ACCEPT 782 packets, 47801 bytes) > pkts bytes target prot opt in out source destination > 1 1400 ppp0_masq all -- * ppp0 0.0.0.0/0 0.0.0.0/0 > 2581 201K eth0.1_masq all -- * eth0.1 0.0.0.0/0 0.0.0.0/0 > > Chain ppp0_masq (1 references) > pkts bytes target prot opt in out source destination > 1 1400 SNAT all -- * * !6.1.3.4 0.0.0.0/0 to:6.1.3.4 > > Chain eth0.1_masq (1 references) > pkts bytes target prot opt in out source destination > 5344 414K MASQUERADE all -- * * 0.0.0.0/0 0.0.0.0/0 > > is the eth0.1 masqing being applied to packets which are routed to the IGS (on > ppp0) provider with the route rule: > > 1002: from 10.75.22.101 lookup IGS > > and associated routing table: > > Table IGS: > > 10.8.0.2 dev tun0 proto kernel scope link src 10.8.0.1 > 192.168.200.1 dev ppp0 scope link src 66.11.173.224 > 10.10.0.0/24 via 10.75.22.1 dev br-lan proto zebra metric 20 equalize > 10.8.0.0/24 via 10.8.0.2 dev tun0 > 192.168.0.0/24 via 10.75.22.5 dev br-lan proto zebra metric 20 equalize > 10.75.22.0/24 dev br-lan proto kernel scope link src 10.75.22.254 > 10.75.23.0/24 via 10.8.0.2 dev tun0 > 192.168.122.0/24 via 10.75.22.151 dev br-lan proto zebra metric 20 equalize > 169.254.0.0/16 via 10.75.22.5 dev br-lan proto zebra metric 20 equalize > default via 192.168.200.1 dev ppp0 src 6.1.3.4 > > Doesn''t the above set the outgoing interface to ppp0 before POSTROUTING is > applied in the iptables "nat" table?I guess I need to be more explicit; I don''t understand it either because from what you have described, either your kernel is badly broken or you are leaving out pertinent information (hint -- ''shorewall dump''). -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ SOLARIS 10 is the OS for Data Centers - provides features such as DTrace, Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW http://p.sf.net/sfu/solaris-dev2dev
Tom Eastep wrote:> Brian J. Murrell wrote: >> Tom Eastep <teastep <at> shorewall.net> writes: >>> If it were not having any effect, the SOURCE IP would be 10.75.22.101, >>> right? >> Ahhh. Yes, of course, you are right. So some SNAT/Masq is happening, just for >> the wrong outgoing interface. >> >>> It''s not clear to me what''s happening from what you have written. >> Sorry. It really is simply a case of using a route_rules entry to force >> traffic from a given IP (10.75.22.101) out through what is normally not the >> "default"/preferred interface (which is provider CGCO), which is working. But >> that the source address of what is normally the preferred/default interface >> (CGCO''s 7.1.7.2) is being masqued to the packets instead of the source address >> of the interface (IGS0 being forced by the route_rules entry. >> >> What I don''t understand is why given the following nat table rules: >> >> Chain POSTROUTING (policy ACCEPT 782 packets, 47801 bytes) >> pkts bytes target prot opt in out source destination >> 1 1400 ppp0_masq all -- * ppp0 0.0.0.0/0 0.0.0.0/0 >> 2581 201K eth0.1_masq all -- * eth0.1 0.0.0.0/0 0.0.0.0/0 >> >> Chain ppp0_masq (1 references) >> pkts bytes target prot opt in out source destination >> 1 1400 SNAT all -- * * !6.1.3.4 0.0.0.0/0 to:6.1.3.4 >> >> Chain eth0.1_masq (1 references) >> pkts bytes target prot opt in out source destination >> 5344 414K MASQUERADE all -- * * 0.0.0.0/0 0.0.0.0/0 >> >> is the eth0.1 masqing being applied to packets which are routed to the IGS (on >> ppp0) provider with the route rule: >> >> 1002: from 10.75.22.101 lookup IGS >> >> and associated routing table: >> >> Table IGS: >> >> 10.8.0.2 dev tun0 proto kernel scope link src 10.8.0.1 >> 192.168.200.1 dev ppp0 scope link src 66.11.173.224 >> 10.10.0.0/24 via 10.75.22.1 dev br-lan proto zebra metric 20 equalize >> 10.8.0.0/24 via 10.8.0.2 dev tun0 >> 192.168.0.0/24 via 10.75.22.5 dev br-lan proto zebra metric 20 equalize >> 10.75.22.0/24 dev br-lan proto kernel scope link src 10.75.22.254 >> 10.75.23.0/24 via 10.8.0.2 dev tun0 >> 192.168.122.0/24 via 10.75.22.151 dev br-lan proto zebra metric 20 equalize >> 169.254.0.0/16 via 10.75.22.5 dev br-lan proto zebra metric 20 equalize >> default via 192.168.200.1 dev ppp0 src 6.1.3.4 >> >> Doesn''t the above set the outgoing interface to ppp0 before POSTROUTING is >> applied in the iptables "nat" table? > > I guess I need to be more explicit; I don''t understand it either because > from what you have described, either your kernel is badly broken or you > are leaving out pertinent information (hint -- ''shorewall dump'').What you are seeing on ppp0 are response packets that are part of connections initiated via DNAT on eth0.1. Here''s a connection from the dump you sent me privately: tcp 6 252 ESTABLISHED src=x.y.y.z dst=6.1.3.4 sport=59075 dport=40718 packets=120 bytes=5929 src=10.75.22.101 dst=y.x.y.z sport=40718 dport=59075 packets=170 bytes=10876 [ASSURED] mark=256 use=2 Outgoing packets from that connection are being routed out of ppp0 by your routing rules. Because those packets are not in the NEW connection state, they do not pass through the ppp0_masq chain so their source IP will be 6.1.3.4. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ SOLARIS 10 is the OS for Data Centers - provides features such as DTrace, Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW http://p.sf.net/sfu/solaris-dev2dev
Tom Eastep <teastep <at> shorewall.net> writes:> > Outgoing packets from that connection are being routed out of ppp0 by > your routing rules. Because those packets are not in the NEW connection > state, they do not pass through the ppp0_masq chain so their source IP > will be 6.1.3.4.Ahhh. Thanx for the analysis Tom. FWIW, I achieved my desired result by abandoning the route_rules entry and instead forcing the use of the desired ISP connection with a tcrules entry that marks the packet for the given ISP: 512:P 10.75.22.101 tcrules is what I likely want in the end anyway given that I will probably try to utilize ipp2p to force the routing of bittorrent packets. Cheers, b. ------------------------------------------------------------------------------ SOLARIS 10 is the OS for Data Centers - provides features such as DTrace, Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW http://p.sf.net/sfu/solaris-dev2dev
Brian J. Murrell wrote:> > tcrules is what I likely want in the end anyway given that I will probably try > to utilize ipp2p to force the routing of bittorrent packets. >That won''t work. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ SOLARIS 10 is the OS for Data Centers - provides features such as DTrace, Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW http://p.sf.net/sfu/solaris-dev2dev
On Wed, 2010-02-17 at 06:20 -0800, Tom Eastep wrote:> > That won''t work.Why''s that? It certainly appears to have worked. b. ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev
Brian J. Murrell wrote:> On Wed, 2010-02-17 at 06:20 -0800, Tom Eastep wrote: >> That won''t work. > > Why''s that? It certainly appears to have worked. >It will work so long as your ISPs are willing to forward outgoing packets with foreign source addresses. A P2P session will begin on one interface or the other; which one determines the external SOURCE and DEST IP addresses. The ipp2p module snoops packet payloads to identify P2P packets; when it identifies one, the session has already been established and all packets sent for the session will be in the ESTABLISHED Netfilter state. So they won''t go through the nat POSTROUTING chain. You will end up with the same situation as earlier in this thread; packets going out through one ISP have the source IP of the other ISP. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Download Intel® Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev