Hi, this might be a dumb question, but I''m not finding much information online. I''m trying to setup a 2.6 linux box to run nat across multiple upstream links as a simple way to aggregate bandwidth. I found the instructions in lartc section 4.2 (http://lartc.org/howto/lartc.rpdb.multiple-links.html) fairly clear and straightforward. I implemented those, and a couple of trivial iptables commands and tried it. Persistent masqueraded connections (like ssh) weren''t very happy. The frequently hung, and I saw the "MASQUERADE: Route sent us somewhere else." error. googling for that, I see lots of suggestions to use connmark, but no examples of how connmark and the multiple link stuff interact. Does anyone have a pointer? The rules I''m using are roughly: /sbin/ip route add P1_NET dev IF1 src IP1 table 201 /sbin/ip route add default via P1 table 201 /sbin/ip route add P1_NET dev IF1 src IP1 /sbin/ip route add 127.0.0.0/8 dev lo table 201 /sbin/ip rule add from IP1 table 201 /sbin/ip route add P2_NET dev IF2 src IP2 table 202 /sbin/ip route add default via P2 table 202 /sbin/ip route add P2_NET dev IF2 src IP2 /sbin/ip route add 127.0.0.0/8 dev lo table 202 /sbin/ip rule add from IP2 table 202 /sbin/ip route del default /sbin/ip route add default scope global nexthop via P1 dev IF1 weight 1 nexthop via P2 dev IF2 weight 4 /sbin/ip route flush cache /sbin/iptables -t nat -A PREROUTING -s LOCALNET -d P1_NET -j ACCEPT /sbin/iptables -t nat -A PREROUTING -s LOCALNET -d P2_NET -j ACCEPT /sbin/iptables -t nat -A POSTROUTING -o IF1 -s LOCALNET -j MASQUERADE /sbin/iptables -t nat -A POSTROUTING -o IF2 -s LOCALNET -j MASQUERADE thanks seph
The problem (as usual) is the change of route the routing box is doing for connections already stablished. Maybe you can try using separate routing tablewith a single internet link for ssh (policy routing). On 1/5/06, seph <seph@directionless.org> wrote:> Hi, this might be a dumb question, but I''m not finding much > information online. > > I''m trying to setup a 2.6 linux box to run nat across multiple > upstream links as a simple way to aggregate bandwidth. I found the > instructions in lartc section 4.2 > (http://lartc.org/howto/lartc.rpdb.multiple-links.html) fairly clear > and straightforward. I implemented those, and a couple of trivial > iptables commands and tried it. > > Persistent masqueraded connections (like ssh) weren''t very happy. The > frequently hung, and I saw the "MASQUERADE: Route sent us somewhere > else." error. googling for that, I see lots of suggestions to use > connmark, but no examples of how connmark and the multiple link stuff > interact. Does anyone have a pointer? > > The rules I''m using are roughly: > > /sbin/ip route add P1_NET dev IF1 src IP1 table 201 > /sbin/ip route add default via P1 table 201 > /sbin/ip route add P1_NET dev IF1 src IP1 > /sbin/ip route add 127.0.0.0/8 dev lo table 201 > /sbin/ip rule add from IP1 table 201 > /sbin/ip route add P2_NET dev IF2 src IP2 table 202 > /sbin/ip route add default via P2 table 202 > /sbin/ip route add P2_NET dev IF2 src IP2 > /sbin/ip route add 127.0.0.0/8 dev lo table 202 > /sbin/ip rule add from IP2 table 202 > > > /sbin/ip route del default > /sbin/ip route add default scope global nexthop via P1 dev IF1 weight 1 nexthop via P2 dev IF2 weight 4 > /sbin/ip route flush cache > > > /sbin/iptables -t nat -A PREROUTING -s LOCALNET -d P1_NET -j ACCEPT > /sbin/iptables -t nat -A PREROUTING -s LOCALNET -d P2_NET -j ACCEPT > /sbin/iptables -t nat -A POSTROUTING -o IF1 -s LOCALNET -j MASQUERADE > /sbin/iptables -t nat -A POSTROUTING -o IF2 -s LOCALNET -j MASQUERADE > > > thanks > seph > _______________________________________________ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >
Witaj Edmundo, W Twoim licie datowanym 5 stycznia 2006 (17:21:52) mo¿na przeczytaæ: Please consult: http://www.ssi.bg/~ja/#routes and my example scpript mpath2.sh published there. Without those patches - it just doesn''t work :)> The problem (as usual) is the change of route the routing box is doing > for connections already stablished.> Maybe you can try using separate routing tablewith a single internet > link for ssh (policy routing).> On 1/5/06, seph <seph@directionless.org> wrote: >> Hi, this might be a dumb question, but I''m not finding much >> information online. >> >> I''m trying to setup a 2.6 linux box to run nat across multiple >> upstream links as a simple way to aggregate bandwidth. I found the >> instructions in lartc section 4.2 >> (http://lartc.org/howto/lartc.rpdb.multiple-links.html) fairly clear >> and straightforward. I implemented those, and a couple of trivial >> iptables commands and tried it. >> >> Persistent masqueraded connections (like ssh) weren''t very happy. The >> frequently hung, and I saw the "MASQUERADE: Route sent us somewhere >> else." error. googling for that, I see lots of suggestions to use >> connmark, but no examples of how connmark and the multiple link stuff >> interact. Does anyone have a pointer? >> >> The rules I''m using are roughly: >> >> /sbin/ip route add P1_NET dev IF1 src IP1 table 201 >> /sbin/ip route add default via P1 table 201 >> /sbin/ip route add P1_NET dev IF1 src IP1 >> /sbin/ip route add 127.0.0.0/8 dev lo table 201 >> /sbin/ip rule add from IP1 table 201 >> /sbin/ip route add P2_NET dev IF2 src IP2 table 202 >> /sbin/ip route add default via P2 table 202 >> /sbin/ip route add P2_NET dev IF2 src IP2 >> /sbin/ip route add 127.0.0.0/8 dev lo table 202 >> /sbin/ip rule add from IP2 table 202 >> >> >> /sbin/ip route del default >> /sbin/ip route add default scope global nexthop via P1 dev IF1 weight 1 nexthop via P2 dev IF2 weight 4 >> /sbin/ip route flush cache >> >> >> /sbin/iptables -t nat -A PREROUTING -s LOCALNET -d P1_NET -j ACCEPT >> /sbin/iptables -t nat -A PREROUTING -s LOCALNET -d P2_NET -j ACCEPT >> /sbin/iptables -t nat -A POSTROUTING -o IF1 -s LOCALNET -j MASQUERADE >> /sbin/iptables -t nat -A POSTROUTING -o IF2 -s LOCALNET -j MASQUERADE >> >> >> thanks >> seph >> _______________________________________________ >> LARTC mailing list >> LARTC@mailman.ds9a.nl >> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >> > _______________________________________________ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc-- Pozdrowienia, Robert Kurjata
I was hoping to avoid having to patch things. I''ll take a look at mpath and see what I want to do. If it really does need patching than I think the howto should be updated to reflect that. seph Robert Kurjata <rkurjata@ire.pw.edu.pl> writes:> Witaj Edmundo, > > W Twoim li�cie datowanym 5 stycznia 2006 (17:21:52) mo¿na przeczytaæ: > > Please consult: http://www.ssi.bg/~ja/#routes and my example scpript > mpath2.sh published there. > > Without those patches - it just doesn''t work :) > > >> The problem (as usual) is the change of route the routing box is doing >> for connections already stablished. > >> Maybe you can try using separate routing tablewith a single internet >> link for ssh (policy routing). > >> On 1/5/06, seph <seph@directionless.org> wrote: >>> Hi, this might be a dumb question, but I''m not finding much >>> information online. >>> >>> I''m trying to setup a 2.6 linux box to run nat across multiple >>> upstream links as a simple way to aggregate bandwidth. I found the >>> instructions in lartc section 4.2 >>> (http://lartc.org/howto/lartc.rpdb.multiple-links.html) fairly clear >>> and straightforward. I implemented those, and a couple of trivial >>> iptables commands and tried it. >>> >>> Persistent masqueraded connections (like ssh) weren''t very happy. The >>> frequently hung, and I saw the "MASQUERADE: Route sent us somewhere >>> else." error. googling for that, I see lots of suggestions to use >>> connmark, but no examples of how connmark and the multiple link stuff >>> interact. Does anyone have a pointer? >>> >>> The rules I''m using are roughly: >>> >>> /sbin/ip route add P1_NET dev IF1 src IP1 table 201 >>> /sbin/ip route add default via P1 table 201 >>> /sbin/ip route add P1_NET dev IF1 src IP1 >>> /sbin/ip route add 127.0.0.0/8 dev lo table 201 >>> /sbin/ip rule add from IP1 table 201 >>> /sbin/ip route add P2_NET dev IF2 src IP2 table 202 >>> /sbin/ip route add default via P2 table 202 >>> /sbin/ip route add P2_NET dev IF2 src IP2 >>> /sbin/ip route add 127.0.0.0/8 dev lo table 202 >>> /sbin/ip rule add from IP2 table 202 >>> >>> >>> /sbin/ip route del default >>> /sbin/ip route add default scope global nexthop via P1 dev IF1 weight 1 nexthop via P2 dev IF2 weight 4 >>> /sbin/ip route flush cache >>> >>> >>> /sbin/iptables -t nat -A PREROUTING -s LOCALNET -d P1_NET -j ACCEPT >>> /sbin/iptables -t nat -A PREROUTING -s LOCALNET -d P2_NET -j ACCEPT >>> /sbin/iptables -t nat -A POSTROUTING -o IF1 -s LOCALNET -j MASQUERADE >>> /sbin/iptables -t nat -A POSTROUTING -o IF2 -s LOCALNET -j MASQUERADE >>> >>> >>> thanks >>> seph >>> _______________________________________________ >>> LARTC mailing list >>> LARTC@mailman.ds9a.nl >>> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc >>> >> _______________________________________________ >> LARTC mailing list >> LARTC@mailman.ds9a.nl >> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc > > > -- > Pozdrowienia, > Robert Kurjata > > _______________________________________________ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
seph wrote:> > I was hoping to avoid having to patch things. I''ll take a look at > mpath and see what I want to do. > > If it really does need patching than I think the howto should be > updated to reflect that. > > sephSeph, The HOWTO has not been updated in years. Who knows when, if ever, it will be? Robert is correct. -- gypsy> Robert Kurjata <rkurjata@ire.pw.edu.pl> writes: > > > Witaj Edmundo, > > > > W Twoim li�cie datowanym 5 stycznia 2006 (17:21:52) mo¿na przeczytaæ: > > > > Please consult: http://www.ssi.bg/~ja/#routes and my example scpript > > mpath2.sh published there. > > > > Without those patches - it just doesn''t work :) > > > > > >> The problem (as usual) is the change of route the routing box is doing > >> for connections already stablished. > > > >> Maybe you can try using separate routing tablewith a single internet > >> link for ssh (policy routing). > > > >> On 1/5/06, seph <seph@directionless.org> wrote: > >>> Hi, this might be a dumb question, but I''m not finding much > >>> information online. > >>> > >>> I''m trying to setup a 2.6 linux box to run nat across multiple > >>> upstream links as a simple way to aggregate bandwidth. I found the > >>> instructions in lartc section 4.2 > >>> (http://lartc.org/howto/lartc.rpdb.multiple-links.html) fairly clear > >>> and straightforward. I implemented those, and a couple of trivial > >>> iptables commands and tried it. > >>> > >>> Persistent masqueraded connections (like ssh) weren''t very happy. The > >>> frequently hung, and I saw the "MASQUERADE: Route sent us somewhere > >>> else." error. googling for that, I see lots of suggestions to use > >>> connmark, but no examples of how connmark and the multiple link stuff > >>> interact. Does anyone have a pointer? > >>> > >>> The rules I''m using are roughly: > >>> > >>> /sbin/ip route add P1_NET dev IF1 src IP1 table 201 > >>> /sbin/ip route add default via P1 table 201 > >>> /sbin/ip route add P1_NET dev IF1 src IP1 > >>> /sbin/ip route add 127.0.0.0/8 dev lo table 201 > >>> /sbin/ip rule add from IP1 table 201 > >>> /sbin/ip route add P2_NET dev IF2 src IP2 table 202 > >>> /sbin/ip route add default via P2 table 202 > >>> /sbin/ip route add P2_NET dev IF2 src IP2 > >>> /sbin/ip route add 127.0.0.0/8 dev lo table 202 > >>> /sbin/ip rule add from IP2 table 202 > >>> > >>> > >>> /sbin/ip route del default > >>> /sbin/ip route add default scope global nexthop via P1 dev IF1 weight 1 nexthop via P2 dev IF2 weight 4 > >>> /sbin/ip route flush cache > >>> > >>> > >>> /sbin/iptables -t nat -A PREROUTING -s LOCALNET -d P1_NET -j ACCEPT > >>> /sbin/iptables -t nat -A PREROUTING -s LOCALNET -d P2_NET -j ACCEPT > >>> /sbin/iptables -t nat -A POSTROUTING -o IF1 -s LOCALNET -j MASQUERADE > >>> /sbin/iptables -t nat -A POSTROUTING -o IF2 -s LOCALNET -j MASQUERADE > >>> > >>> > >>> thanks > >>> seph > > -- > > Pozdrowienia, > > Robert Kurjata
I read the previous thread about this but I seem to have a small problem. I''m running gentoo with 2.6.14 kernel and have applied the patch from http://www.ssi.bg/~ja/#routes. If I try to lynx to two different pages from the box itself it switches the interfaces every other time which is how it''s supposed to work. If I use an extra machine and set it''s gateway to 10.0.4.211(eth0), then all requests are being sent to just one interface all the time. The same happens if I try other machines. It''s only requests generated inside the box which are being routed properly. I''m just wondering if something is amiss in my setup. I have also tried the mpath2.sh script, but I got the same results. My setup is like this: eth0: 10.0.4.211 (internal network) eth1: 192.168.1.2 (ISP1) eth2: 10.0.0.2 (ISP2) Commands: ip route add 192.168.1.0/24 dev eth1 src 192.168.1.2 table T1 ip route add default via 192.168.1.1 table T1 ip route add 10.0.0.0/24 dev eth2 src 10.0.0.2 table T2 ip route add default via 10.0.0.1 table T2 ip rule add from 192.168.1.2 table T1 ip rule add from 10.0.0.2 table T2 ip route add default scope global nexthop via 192.168.1.1 dev eth1 weight 1 nexthop via 10.0.0.1 dev eth2 weight 1 iptables -t nat -A PREROUTING -s 10.0.4.0/255.255.254.0 -d 192.168.1.0/24 -j ACCEPT iptables -t nat -A PREROUTING -s 10.0.4.0/255.255.254.0 -d 10.0.0.0/24 -j ACCEPT iptables -t nat -A POSTROUTING -o eth1 -s 10.0.4.0/255.255.254.0 -j MASQUERADE iptables -t nat -A POSTROUTING -o eth2 -s 10.0.4.0/255.255.254.0 -j MASQUERADE