how to use iproute/iptables to balance by packet OUTBOUND traffic between 2 PPP (pptp cable modem and pppoe adsl modem) links to the same ISP . also by using the fact that the ISP dosen''t filter source IP address - meaning i can use either PPP''s link assigned IP address as the source IP in the ip header.., it works for both devices.., what i want to achive is : when uploading a large file , i could use both devices to shoot data but bound only to one source IP. Naturally i could expect all data to go back through the device with that corresponding IP , atleast that is what happens according to ethereal _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
junk@maild.mine.nu wrote:> how to use iproute/iptables to balance by packet OUTBOUND traffic between > 2 PPP (pptp cable modem and pppoe adsl modem) links to the same ISP . also > by using the fact that the ISP dosen''t filter source IP address - meaning > i can use either PPP''s link assigned IP address as the source IP in the ip > header.., it works for both devices.., what i want to achive is : when > uploading a large file , i could use both devices to shoot data but bound > only to one source IP. > > Naturally i could expect all data to go back through the device with that > corresponding IP , atleast that is what happens according to etherealQuick copy n past of a reply I gave recently. Another way you could look into is doing it at IP level using the netfilter patch Nth. You could just round robin packets over the links - not as nice as multilink as packet size isn''t accounted for, but may be OK in practise. Below is the help for it - You may not need to do the NAT but AIUI you could use it to mark and then route using the marks. Andy. Author: Fabrice MARIE <fabrice@netfilter.org> Status: Works For Me. This option adds CONFIG_IP_NF_MATCH_NTH, which supplies a match module that will allow you to match every Nth packet encountered. By default there are 16 different counters that can be used. This match functions in one of two ways 1) Match ever Nth packet, and only the Nth packet. example: iptables -t mangle -A PREROUTING -m nth --every 10 -j DROP This rule will drop every 10th packet. 2) Unique rule for every packet. This is an easy and quick method to produce load-balancing for both inbound and outbound. example: iptables -t nat -A POSTROUTING -o eth0 -m nth --counter 7 \ --every 3 --packet 0 -j SNAT --to-source 10.0.0.5 iptables -t nat -A POSTROUTING -o eth0 -m nth --counter 7 \ --every 3 --packet 1 -j SNAT --to-source 10.0.0.6 iptables -t nat -A POSTROUTING -o eth0 -m nth --counter 7 \ --every 3 --packet 2 -j SNAT --to-source 10.0.0.7 This example evenly splits connections between the three SNAT addresses. By using the mangle table and iproute2, you can setup complex load-balanced routing. There''s lot of other uses. Be creative! Suppported options are: --every Nth Match every Nth packet [--counter] num Use counter 0-15 (default:0) [--start] num Initialize the counter at the number ''num'' instead of 0. Must be between 0 and Nth-1 [--packet] num Match on ''num'' packet. Must be between 0 and Nth-1. If --packet is used for a counter than there must be Nth number of --packet rules, covering all values between 0 and Nth-1 inclusively. Andy. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
I am still looking for a solution to this problem.......> junk@maild.mine.nu wrote: >> Hi, >> Yes i did give this a try a couple of times before with no success >> >> /sbin/iptables -I OUTPUT -m nth --every 2 --packet 1 -t mangle -j MARK >> --set-mark 0x2 >> /sbin/iptables -I OUTPUT -m nth --every 2 --packet 0 -t mangle -j MARK >> --set-mark 0x1 >> >> ip rule : >> 0: from all lookup local >> 201: from all fwmark 0x2 lookup 202 >> 201: from all fwmark 0x1 lookup 201 >> 32766: from all lookup main >> ip route show table 202 >> default via 212.199.28.244 dev ppp1 proto static src 80.178.89.120 >> ip route show table 201 >> default via 212.199.26.111 dev ppp0 proto static src 84.94.148.214 >> ip route show table main >> 212.199.26.111 dev ppp0 proto kernel scope link src 84.94.148.214 >> 212.199.28.244 dev ppp1 proto kernel scope link src 80.178.89.120 >> default proto static equalize >> nexthop via 212.199.26.111 dev ppp0 weight 1 >> nexthop via 212.199.28.244 dev ppp1 weight 1 > > > > I must admit I am more into tc than ip so I''ve never tried it. > > My guess is that you need to get rid of equalize/weights as these load > balance per connection and the routes get cached. > > Andy. > >how to use iproute/iptables to balance by packet OUTBOUND traffic between 2 PPP (pptp cable modem and pppoe adsl modem) links to the same ISP . also by using the fact that the ISP dosen''t filter source IP address - meaning i can use either PPP''s link assigned IP address as the source IP in the ip header.., it works for both devices.., what i want to achive is : when uploading a large file , i could use both devices to shoot data but bound only to one source IP. Naturally i could expect all data to go back through the device with that corresponding IP , atleast that is what happens according to ethereal how to use iproute/iptables to balance by packet OUTBOUND traffic between 2 PPP (pptp cable modem and pppoe adsl modem) links to the same ISP . also by using the fact that the ISP dosen''t filter source IP address - meaning i can use either PPP''s link assigned IP address as the source IP in the ip header.., it works for both devices.., what i want to achive is : when uploading a large file , i could use both devices to shoot data but bound only to one source IP. Naturally i could expect all data to go back through the device with that corresponding IP , atleast that is what happens according to ethereal _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
junk@maild.mine.nu wrote:> > I am still looking for a solution to this problem....... > > junk@maild.mine.nu wrote:I realize this is the opposite of what you try to accomplish, but perhaps it will help? Google "LARTC load balance a file download across two connections - success" gypsy _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hey thanks for replying , i got this from ethereal ..: tethereal -i ppp0 Capturing on ppp0 0.000000 80.178.69.24 -> 67.15.162.64 TCP 31604 > 53880 [ACK] Seq=0 Ack=0 Win=5680 Len=1360 0.201730 80.178.69.24 -> 67.15.162.64 TCP [TCP Previous segment lost] 31604 > 53880 [ACK] Seq=2720 Ack=0 Win=5680 Len=1360 0.301142 80.178.69.24 -> 67.15.162.64 TCP [TCP Previous segment lost] 31604 > 53880 [PSH, ACK] Seq=5440 Ack=0 Win=5680 Len=1360 0.417813 80.178.69.24 -> 67.15.162.64 TCP [TCP Previous segment lost] 31604 > 53880 [ACK] Seq=8160 Ack=0 Win=5680 Len=1360 0.516307 80.178.69.24 -> 67.15.162.64 TCP [TCP Previous segment lost] 31604 > 53880 [ACK] Seq=10880 Ack=0 Win=5680 Len=1360 0.599234 80.178.69.24 -> 67.15.162.64 TCP [TCP Previous segment lost] 31604 > 53880 [ACK] Seq=13600 Ack=0 Win=5680 Len=1360 0.647668 80.178.69.24 -> 67.15.162.64 TCP [TCP Previous segment lost] 31604 > 53880 [ACK] Seq=16320 Ack=0 Win=5680 Len=1360 0.699314 80.178.69.24 -> 67.15.162.64 TCP [TCP Previous segment lost] .... tethereal -i ppp1 Capturing on ppp1 0.000000 67.15.162.64 -> 80.178.69.24 TCP 53880 > 31604 [ACK] Seq=0 Ack=0 Win=44880 Len=0 SLE=3114001847 SRE=3114003207 SLE=3113999127 SRE=3114000487 SLE=3113996407 SRE=3113997767 SLE=3113993687 SRE=3113995047 0.008278 80.178.69.24 -> 67.15.162.64 TCP [TCP Previous segment lost] 31604 > 53880 [PSH, ACK] Seq=43520 Ack=0 Win=5680 Len=1360 0.014025 67.15.162.64 -> 80.178.69.24 TCP [TCP Dup ACK 1#1] 53880 > 31604 [ACK] Seq=0 Ack=0 Win=46240 Len=0 SLE=3114004567 SRE=3114005927 SLE=3114001847 SRE=3114003207 SLE=3113999127 SRE=3114000487 SLE=3113996407 SRE=3113997767 0.091375 67.15.162.64 -> 80.178.69.24 TCP [TCP Dup ACK 1#2] 53880 > 31604 [ACK] Seq=0 Ack=0 Win=46240 Len=0 SLE=3114007287 SRE=3114008647 SLE=3114004567 SRE=3114005927 SLE=3114001847 SRE=3114003207 SLE=3113999127 SRE=3114000487 0.098127 67.15.162.64 -> 80.178.69.24 TCP 53880 > 31604 [ACK] Seq=0 Ack=2720 Win=43520 Len=0 SLE=3114007287 SRE=3114008647 SLE=3114004567 SRE=3114005927 SLE=3114001847 SRE=3114003207 SLE=3113999127 SRE=3114000487 0.175701 67.15.162.64 -> 80.178.69.24 TCP [TCP Dup ACK 5#1] 53880 > 31604 [ACK] Seq=0 Ack=2720 Win=44880 Len=0 SLE=3114010007 SRE=3114011367 SLE=3114007287 SRE=3114008647 SLE=3114004567 SRE=3114005927 SLE=3114001847 SRE=3114003207 0.175892 80.178.69.24 -> 67.15.162.64 TCP [TCP Previous segment lost] 31604 > 53880 [ACK] Seq=46240 Ack=0 Win=5680 Len=1360 ..... Combine nth match with ROUTE target and you will get it. ip ro re default dev ppp0 iptables -A POSTROUTING -t mangle -o ppp+ -m nth --every 2 --packet 0 -j ROUTE --oif ppp0 iptables -A POSTROUTING -t mangle -o ppp+ -m nth --every 2 --packet 1 -j ROUTE --oif ppp1 iptables -A POSTROUTING -t nat -o ppp+ -j SNAT --to $ppp0_ip_addr I just guess that it''ll work, I don''t have IP spoofing enabled lines to test I am still looking for a solution to this problem.......> junk@maild.mine.nu wrote: >> Hi, >> Yes i did give this a try a couple of times before with no success >> >> /sbin/iptables -I OUTPUT -m nth --every 2 --packet 1 -t mangle -j MARK >> --set-mark 0x2 >> /sbin/iptables -I OUTPUT -m nth --every 2 --packet 0 -t mangle -j MARK >> --set-mark 0x1 >> >> ip rule : >> 0: from all lookup local >> 201: from all fwmark 0x2 lookup 202 >> 201: from all fwmark 0x1 lookup 201 >> 32766: from all lookup main >> ip route show table 202 >> default via 212.199.28.244 dev ppp1 proto static src 80.178.89.120 iproute show table 201>> default via 212.199.26.111 dev ppp0 proto static src 84.94.148.214 iproute show table main>> 212.199.26.111 dev ppp0 proto kernel scope link src 84.94.148.214212.199.28.244 dev ppp1 proto kernel scope link src 80.178.89.120 default proto static equalize>> nexthop via 212.199.26.111 dev ppp0 weight 1 >> nexthop via 212.199.28.244 dev ppp1 weight 1 > > > > I must admit I am more into tc than ip so I''ve never tried it. > > My guess is that you need to get rid of equalize/weights as these loadbalance per connection and the routes get cached.> > Andy. > >how to use iproute/iptables to balance by packet OUTBOUND traffic between 2 PPP (pptp cable modem and pppoe adsl modem) links to the same ISP . also by using the fact that the ISP dosen''t filter source IP address - meaning i can use either PPP''s link assigned IP address as the source IP in the ip header.., it works for both devices.., what i want to achive is : when uploading a large file , i could use both devices to shoot data but bound only to one source IP. Naturally i could expect all data to go back through the device with that corresponding IP , atleast that is what happens according to ethereal _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
junk@maild.mine.nu wrote:> I am still looking for a solution to this problem....... > >>junk@maild.mine.nu wrote: >> >>>Hi, >>>Yes i did give this a try a couple of times before with no success >>> >>> /sbin/iptables -I OUTPUT -m nth --every 2 --packet 1 -t mangle -j MARK >>>--set-mark 0x2 >>> /sbin/iptables -I OUTPUT -m nth --every 2 --packet 0 -t mangle -j MARK >>>--set-mark 0x1 >>> >>> ip rule : >>>0: from all lookup local >>>201: from all fwmark 0x2 lookup 202 >>>201: from all fwmark 0x1 lookup 201 >>>32766: from all lookup main >>>ip route show table 202 >>>default via 212.199.28.244 dev ppp1 proto static src 80.178.89.120 >>>ip route show table 201 >>>default via 212.199.26.111 dev ppp0 proto static src 84.94.148.214 >>>ip route show table main >>>212.199.26.111 dev ppp0 proto kernel scope link src 84.94.148.214 >>>212.199.28.244 dev ppp1 proto kernel scope link src 80.178.89.120 >>>default proto static equalize >>> nexthop via 212.199.26.111 dev ppp0 weight 1 >>> nexthop via 212.199.28.244 dev ppp1 weight 1 >> >> >> >>I must admit I am more into tc than ip so I''ve never tried it. >> >>My guess is that you need to get rid of equalize/weights as these load >>balance per connection and the routes get cached. >> >>Andy.More things I would try - mark in postrouting, snat to the adress of downlink you want return to use. Get rid of the src adresses in in tables. Andy. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Andy Furniss wrote:> mark in postrouting,Oops - I mean mark in forward aswell as output - do snat in postrouting nat table. Andy. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
junk@maild.mine.nu wrote:> I am still looking for a solution to this problem....... > >>junk@maild.mine.nu wrote: >> >>>Hi, >>>Yes i did give this a try a couple of times before with no success >>> >>> /sbin/iptables -I OUTPUT -m nth --every 2 --packet 1 -t mangle -j MARK >>>--set-mark 0x2 >>> /sbin/iptables -I OUTPUT -m nth --every 2 --packet 0 -t mangle -j MARK >>>--set-mark 0x1Had a go at this on 2.6.9 / iptables 1.2.11 and it seeems nth is broken. I''ll try again tomorrow. Andy. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Andy Furniss
2005-Jan-13 12:27 UTC
Re: [LARTC] How to balance OUTBOUND traffic by packet if..
Andy Furniss wrote:> junk@maild.mine.nu wrote: > >> I am still looking for a solution to this problem....... >> >>> junk@maild.mine.nu wrote: >>> >>>> Hi, >>>> Yes i did give this a try a couple of times before with no success >>>> >>>> /sbin/iptables -I OUTPUT -m nth --every 2 --packet 1 -t mangle -j MARK >>>> --set-mark 0x2 >>>> /sbin/iptables -I OUTPUT -m nth --every 2 --packet 0 -t mangle -j MARK >>>> --set-mark 0x1 > > > Had a go at this on 2.6.9 / iptables 1.2.11 and it seeems nth is broken. > > I''ll try again tomorrow.Seems still broken. I built vanilla(apart from nth) 2.6.10, new iptables 1.2.11 + pom-200400621 with runme extra only said y to nth. I see - Chain OUTPUT (policy ACCEPT 817 packets, 103K bytes) pkts bytes target prot opt in out source destination 15 1260 MARK all -- * eth0 0.0.0.0/0 0.0.0.0/0 every 2th packet #0 MARK set 0x1 15 1260 MARK all -- * eth0 0.0.0.0/0 0.0.0.0/0 every 2th packet #1 MARK set 0x2 Chain POSTROUTING (policy ACCEPT 817 packets, 103K bytes) pkts bytes target prot opt in out source destination 0 0 all -- * eth0 0.0.0.0/0 0.0.0.0/0 MARK match 0x1 15 1260 all -- * eth0 0.0.0.0/0 0.0.0.0/0 MARK match 0x2 [root@amd /home/andy]# cat ./nth-test iptables -t mangle -A OUTPUT -o eth0 -m nth --counter 1 --every 2 --packet 0 -j MARK --set-mark 1 iptables -t mangle -A OUTPUT -o eth0 -m nth --counter 1 --every 2 --packet 1 -j MARK --set-mark 2 iptables -t mangle -A POSTROUTING -o eth0 -m mark --mark 1 iptables -t mangle -A POSTROUTING -o eth0 -m mark --mark 2 Any Ideas anyone? TIA Andy.
Patrick McHardy
2005-Jan-14 02:08 UTC
Re: [LARTC] How to balance OUTBOUND traffic by packet if..
Andy Furniss wrote:> Seems still broken. > > I built vanilla(apart from nth) 2.6.10, new iptables 1.2.11 + > pom-200400621 with runme extra only said y to nth. > > I see - > > Chain OUTPUT (policy ACCEPT 817 packets, 103K bytes) > pkts bytes target prot opt in out source destination > 15 1260 MARK all -- * eth0 0.0.0.0/0 > 0.0.0.0/0 every 2th packet #0 MARK set 0x1 > 15 1260 MARK all -- * eth0 0.0.0.0/0 > 0.0.0.0/0 every 2th packet #1 MARK set 0x2 > > Chain POSTROUTING (policy ACCEPT 817 packets, 103K bytes) > pkts bytes target prot opt in out source destination > 0 0 all -- * eth0 0.0.0.0/0 > 0.0.0.0/0 MARK match 0x1 > 15 1260 all -- * eth0 0.0.0.0/0 > 0.0.0.0/0 MARK match 0x2 > [root@amd /home/andy]# cat ./nth-test > iptables -t mangle -A OUTPUT -o eth0 -m nth --counter 1 --every 2 > --packet 0 -j MARK --set-mark 1 > iptables -t mangle -A OUTPUT -o eth0 -m nth --counter 1 --every 2 > --packet 1 -j MARK --set-mark 2 > iptables -t mangle -A POSTROUTING -o eth0 -m mark --mark 1 > iptables -t mangle -A POSTROUTING -o eth0 -m mark --mark 2 > > Any Ideas anyone?Try adding "-m mark --mark X -j RETURN" rules after the MARK rules. The MARK target is non-terminal, so you are overwriting the mark in the second rule. Alternatively you can use "--start 1" in the one of the nth matches. Regards Patrick
Patrick McHardy wrote:> Andy Furniss wrote: > >> Seems still broken. >> >> I built vanilla(apart from nth) 2.6.10, new iptables 1.2.11 + >> pom-200400621 with runme extra only said y to nth. >> >> I see - >> >> Chain OUTPUT (policy ACCEPT 817 packets, 103K bytes) >> pkts bytes target prot opt in out source destination >> 15 1260 MARK all -- * eth0 0.0.0.0/0 >> 0.0.0.0/0 every 2th packet #0 MARK set 0x1 >> 15 1260 MARK all -- * eth0 0.0.0.0/0 >> 0.0.0.0/0 every 2th packet #1 MARK set 0x2 >> >> Chain POSTROUTING (policy ACCEPT 817 packets, 103K bytes) >> pkts bytes target prot opt in out source destination >> 0 0 all -- * eth0 0.0.0.0/0 >> 0.0.0.0/0 MARK match 0x1 >> 15 1260 all -- * eth0 0.0.0.0/0 >> 0.0.0.0/0 MARK match 0x2 >> [root@amd /home/andy]# cat ./nth-test >> iptables -t mangle -A OUTPUT -o eth0 -m nth --counter 1 --every 2 >> --packet 0 -j MARK --set-mark 1 >> iptables -t mangle -A OUTPUT -o eth0 -m nth --counter 1 --every 2 >> --packet 1 -j MARK --set-mark 2 >> iptables -t mangle -A POSTROUTING -o eth0 -m mark --mark 1 >> iptables -t mangle -A POSTROUTING -o eth0 -m mark --mark 2 >> >> Any Ideas anyone? > > > > Try adding "-m mark --mark X -j RETURN" rules after the MARK rules. The > MARK > target is non-terminal, so you are overwriting the mark in the second rule. > Alternatively you can use "--start 1" in the one of the nth matches.Thanks Patrick - I assumed that with the same counter and packet 0/1 that nth would only actually match alternate packets (start didn''t work). So this works - iptables -t mangle -A OUTPUT -o eth0 -m nth --counter 1 --every 2 --packet 0 -j MARK --set-mark 1 iptables -t mangle -A OUTPUT -o eth0 -m mark --mark 1 -j RETURN iptables -t mangle -A OUTPUT -o eth0 -m nth --counter 1 --every 2 --packet 1 -j MARK --set-mark 2 iptables -t mangle -A OUTPUT -o eth0 -m mark --mark 2 -j RETURN but is over complicated really - Why I didn''t do it like below in the first place I don''t know <slaps self> iptables -t mangle -A OUTPUT -o eth0 -j MARK --set-mark 1 iptables -t mangle -A OUTPUT -o eth0 -m nth --every 2 -j MARK --set-mark 2 Andy. PS had to remove junk@maild.mine.nu from CC as my ISP server threw a domain not found - can someone else CC him please. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/