LARTC - Aug 2006 - Routing packets over multiple links (NICS) all on the same ISP all with same gateway.

Ok ive been trying to get this to work for about half a year now. Ive
searched all over the internet for a solution for
my  problem. Ive found some solutions, but they only led me to yet more
problems.

What we want to do is the following:
I live in a student complex with 7 other people. Every room has its own
internet connection from the same ISP.
Ip, gateway, subnet are asigned through dhcp on mac-adres basis. Every
internet connection is capped at 20mbit
up/down.
We want to get all computers on an internal network. So we need some sort of
router that accepts all 8 internet
connections and routes it out over a gbit nic -> switch -> internal
network
So we have a computer with 10 nics and "hopefully" enough internal
bandwith.

Attempt 1
I read that FreeBSD was the choice for network/router/servers. So after alot
of problems installing FreeBSD
i found out that FreeBSD doesnt support multiple gateway routing to start
with.

Attempt 2
 Ive tried the nano guide http://www.ssi.bg/~ja/nano.txt with the julian
patches on FC5 2.6 kernel. The problem
there was that it only used 1 external nic. I think because
all external connections had the same gateway. It didnt
handle the routing like it supposed to do.

Attempt 3:
Ive tried to follow the following guide
http://www.leglug.org/node/Load%20Balancing%20Across%20Multiple%20Links
I first tried to use this script in a 2.6 kernel (no julian
patch, ip_route_multipath_cache enabled) It could not find any match for the
"-m random --average 50 " function. So after trying al kinds of things
and
searching the net a
lot, i found out that 2.6 kernel doesnt support this random match??
Anyway, so ive installed Slackware 10.2 on the box with a 2.4.32 kernel.
Ofcourse with multipath routing enabled
in the kernel. Again it couldnt: load match ''random:
/usr/lib/iptables/libipt_random.so''
As far as i know it should be available in iptables 1.3.3 that comes with
slackware 10.2.

I know that trunking/bonding the 8 nics with the swicht in the complex would
be the ideal solution, but first of all
 the isp won''t do this, and seccond im not even sure if the switch
supports
this (if i could even get acces to it)

Some help would be greatly apreciated.


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

On Monday 14 August 2006 12:42, Jacques Rompen wrote:
> Ok ive been trying to get this to work for about half a year now. Ive
> searched all over the internet for a solution for
> my  problem. Ive found some solutions, but they only led me to yet more
> problems.
>
> What we want to do is the following:
> I live in a student complex with 7 other people. Every room has its own
> internet connection from the same ISP.
> Ip, gateway, subnet are asigned through dhcp on mac-adres basis. Every
> internet connection is capped at 20mbit
> up/down.
> We want to get all computers on an internal network. So we need some sort
> of router that accepts all 8 internet
> connections and routes it out over a gbit nic -> switch -> internal
network
> So we have a computer with 10 nics and "hopefully" enough
internal
> bandwith.
one simple solution is in this post[1] (read all the thread if you whant a 
context). Bassically it uses iptables CONNMARK to remembre to which iface(and 
certanly wich src ip) each connection flow belongs. 

You will have to use same gw in all ifaces, maybe the ip route
"onlink"
ption(command line) make your script simpler. No need to use julian patch and 
certanly DO NOT USE CONFIG_IP_ROUTE_MULTIPATH_CACHED in your kernel config.

Feel free to post any doubts.

Greets.

[1]http://mailman.ds9a.nl/pipermail/lartc/2006q2/018964.html
-- 
--
Luciano

On 14-08-2006 17:42, Jacques Rompen wrote:
...
> What we want to do is the following:
> I live in a student complex with 7 other people. Every room has its own 
> internet connection from the same ISP.
> Ip, gateway, subnet are asigned through dhcp on mac-adres basis. Every 
> internet connection is capped at 20mbit
> up/down.
> We want to get all computers on an internal network. So we need some 
> sort of router that accepts all 8 internet
> connections and routes it out over a gbit nic -> switch -> internal
network
> So we have a computer with 10 nics and "hopefully" enough
internal bandwith.
Maybe you have some other reasons to make one gateway but to have 
internal network you only need to set a second address from local 
range (not colliding with that used by ISP) on each box.

But if you really need this together and with multipath, setting 
CONFIG_IP_ROUTE_MULTIPATH = "y" and 
CONFIG_IP_ROUTE_MULTIPATH_CACHED = "n" (!) should change 
multipath randomly.

Jarek P.

On 8/16/06, Jarek Poplawski <jarkao2@o2.pl> wrote:
>
> On 14-08-2006 17:42, Jacques Rompen wrote:
> ...
> > What we want to do is the following:
> > I live in a student complex with 7 other people. Every room has its
own
> > internet connection from the same ISP.
> > Ip, gateway, subnet are asigned through dhcp on mac-adres basis. Every
> > internet connection is capped at 20mbit
> > up/down.
> > We want to get all computers on an internal network. So we need some
> > sort of router that accepts all 8 internet
> > connections and routes it out over a gbit nic -> switch ->
internal
> network
> > So we have a computer with 10 nics and "hopefully" enough
internal
> bandwith.
>
> Maybe you have some other reasons to make one gateway but to have
> internal network you only need to set a second address from local
> range (not colliding with that used by ISP) on each box.
>
> But if you really need this together and with multipath, setting
> CONFIG_IP_ROUTE_MULTIPATH = "y" and
> CONFIG_IP_ROUTE_MULTIPATH_CACHED = "n" (!) should change
> multipath randomly.
>
> Jarek P.
>
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
The 2 ips on every box wouldnt work because: Most people here are complete
network newbies, we want to connect more then 8 computers (most people have
a laptop and pc), we would only have a 20mbit internal network.

Im a bit confused. In the 2.4 kernel there issent even an option
ip_route_multipath_cached, only ip_route_multipath. Or am i mistaking here?
If there issent, then I couldnt have put it to ''Y'' so why
didnt the random
function work?

In the 2.6 kernel there is a option ip_route_multipath and
ip_route_multipath_cached. But if i dont say ''Y'' to
ip_route_multipatch_cached i cant say yes to round_robin, random_match etc.
I thought i needed that option random_match set to ''y''. (im
not sure if its
exactly spelled like that, but i''m talking about the four or five sub
options to ip_route_multipath_cached)


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

On 8/14/06, Luciano Ruete <luciano@lugmen.org.ar> wrote:
> On Monday 14 August 2006 12:42, Jacques Rompen wrote:
> > Ok ive been trying to get this to work for about half a year now. Ive
> > searched all over the internet for a solution for
> > my  problem. Ive found some solutions, but they only led me to yet
more
> > problems.
> >
> > What we want to do is the following:
> > I live in a student complex with 7 other people. Every room has its
own
> > internet connection from the same ISP.
> > Ip, gateway, subnet are asigned through dhcp on mac-adres basis. Every
> > internet connection is capped at 20mbit
> > up/down.
> > We want to get all computers on an internal network. So we need some
> sort
> > of router that accepts all 8 internet
> > connections and routes it out over a gbit nic -> switch ->
internal
> network
> > So we have a computer with 10 nics and "hopefully" enough
internal
> > bandwith.
>
> one simple solution is in this post[1] (read all the thread if you whant a
> context). Bassically it uses iptables CONNMARK to remembre to which
> iface(and
> certanly wich src ip) each connection flow belongs.
>
> You will have to use same gw in all ifaces, maybe the ip route
"onlink"
> ption(command line) make your script simpler. No need to use julian patch
> and
> certanly DO NOT USE CONFIG_IP_ROUTE_MULTIPATH_CACHED in your kernel
> config.
>
> Feel free to post any doubts.
>
> Greets.
>
> [1]http://mailman.ds9a.nl/pipermail/lartc/2006q2/018964.html
> --
> --
> Luciano

Hi Luciano,

Ive tried the above solution but then with juliano''s patch instead of
conmark. It didnt work then because this part of the script:

ip ro add default  nexthop via x.x.x.x dev eth1 weight 1 nexthop via
y.y.y.y dev eth2
ip route add default table provider1 via x.x.x.x dev eth1
ip route add default table provider2 via y.y.y.y dev eth2
# and most important
ip rule add fwmark 0x1 table provider1
ip rule add fwmark 0x2 table provider2

somehow looked at the default gateway x.x.x.x or y.y.y.y instead of to the
"dev ethx"
Because the gateway adresses were both the same x.x.x.x = y.y.y.y all the
outgoing packets were routed out over eth1.
Im not totaly sure, but the conmark line''s only make sure that packets
that
go out over some nic get back over the same nic? Or do they infact fix the
"same gateway"problem?

This is why i tried to get the script from my first post with random match
to work. This guy does exactly what i need to do, but somehow i can''t
get
random match to work. You think i should use a 2.6 or a 2.4 kernel?

Regards and thanks,

Jacques


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

On 16-08-2006 14:50, Jacques Rompen wrote:
...
> The 2 ips on every box wouldnt work because: Most people here are 
> complete network newbies, we want to connect more then 8 computers (most 
> people have a laptop and pc), we would only have a 20mbit internal network.
>  
> Im a bit confused. In the 2.4 kernel there issent even an option 
> ip_route_multipath_cached, only ip_route_multipath. Or am i mistaking here?
> If there issent, then I couldnt have put it to ''Y'' so why
didnt the
> random function work?
>  
> In the 2.6 kernel there is a option ip_route_multipath and 
> ip_route_multipath_cached. But if i dont say ''Y'' to 
> ip_route_multipatch_cached i cant say yes to round_robin, random_match 
> etc. I thought i needed that option random_match set to
''y''. (im not
> sure if its exactly spelled like that, but i''m talking about the
four or
> five sub options to ip_route_multipath_cached)
As far as I know multipath cached should work for routing output 
of a current box but it''s not working for router (if something 
has not changed lately). Multipath without "cached" should do 
simple randomization, but maybe for the begining it would be 
better to assign fixed paths by source ip and save experimenting 
with multipaths for later.

Jarek P.

On 8/16/06, Jarek Poplawski <jarkao2@o2.pl> wrote:
>
> As far as I know multipath cached should work for routing output
> of a current box but it''s not working for router (if something
> has not changed lately). Multipath without "cached" should do
> simple randomization, but maybe for the begining it would be
> better to assign fixed paths by source ip and save experimenting
> with multipaths for later.
>
> Jarek P.

Hi Jarek,

I will try with only multipath.

Ive tried the static routes. But still ive had trouble getting this code to
work
ip ro add default  nexthop via x.x.x.x dev eth1 weight 1 nexthop via
y.y.y.y dev eth2
ip route add default table provider1 via x.x.x.x dev eth1
ip route add default table provider2 via y.y.y.y dev eth2
because x.x.x.x = y.y.y.y in my case. Somehow it doesnt look at the eth
device anymore.


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

On Wednesday 16 August 2006 10:11, Jacques Rompen wrote:
> On 8/14/06, Luciano Ruete <luciano@lugmen.org.ar> wrote:
> > On Monday 14 August 2006 12:42, Jacques Rompen wrote:
> > > Ok ive been trying to get this to work for about half a year now.
Ive
> > > searched all over the internet for a solution for
> > > my  problem. Ive found some solutions, but they only led me to
yet more
> > > problems.
> > >
> > > What we want to do is the following:
> > > I live in a student complex with 7 other people. Every room has
its own
> > > internet connection from the same ISP.
> > > Ip, gateway, subnet are asigned through dhcp on mac-adres basis.
Every
> > > internet connection is capped at 20mbit
> > > up/down.
> > > We want to get all computers on an internal network. So we need
some
> >
> > sort
> >
> > > of router that accepts all 8 internet
> > > connections and routes it out over a gbit nic -> switch ->
internal
> >
> > network
> >
> > > So we have a computer with 10 nics and "hopefully"
enough internal
> > > bandwith.
> >
> > one simple solution is in this post[1] (read all the thread if you
whant
> > a context). Bassically it uses iptables CONNMARK to remembre to which
> > iface(and
> > certanly wich src ip) each connection flow belongs.
> >
> > You will have to use same gw in all ifaces, maybe the ip route
"onlink"
> > ption(command line) make your script simpler. No need to use julian
patch
> > and
> > certanly DO NOT USE CONFIG_IP_ROUTE_MULTIPATH_CACHED in your kernel
> > config.
> >
> > Feel free to post any doubts.
> >
> > Greets.
> >
> > [1]http://mailman.ds9a.nl/pipermail/lartc/2006q2/018964.html
> > --
> > --
> > Luciano
>
> Hi Luciano,
>
> Ive tried the above solution but then with juliano''s patch instead
of
> conmark. It didnt work then because this part of the script:
>
> ip ro add default  nexthop via x.x.x.x dev eth1 weight 1 nexthop via
> y.y.y.y dev eth2
> ip route add default table provider1 via x.x.x.x dev eth1
> ip route add default table provider2 via y.y.y.y dev eth2
> # and most important
> ip rule add fwmark 0x1 table provider1
> ip rule add fwmark 0x2 table provider2
>
> somehow looked at the default gateway x.x.x.x or y.y.y.y instead of to the
> "dev ethx"
> Because the gateway adresses were both the same x.x.x.x = y.y.y.y all the
> outgoing packets were routed out over eth1.
ok, what if you try with onlink:
ip ro add default  nexthop via x.x.x.x eth1 onlink nexthop via x.x.x.x eth2 
onlink and so on...
> Im not totaly sure, but the conmark line''s only make sure that
packets that
> go out over some nic get back over the same nic? Or do they infact fix the
> "same gateway"problem?
You are rigth, connmark does not solve the "same gw" problem, connmark
is to
guarantice that packet that belongs to a session dialog keeps the same src 
address. There is no need to patch the kernel, and it works perfectly well. 
> This is why i tried to get the script from my first post with random match
> to work. This guy does exactly what i need to do, but somehow i
can''t get
> random match to work. You think i should use a 2.6 or a 2.4 kernel?
I heavly recommend you to use a 2.6 CONNMARK enabled kernel, without
julian''s
patchset and without setting CONFIG_IP_ROUTE_MULTIPATH_CACHED (turn it off) 
in my experience this is the simplest and robust solution.

Iptables random module comes with 2.6 stock kernel(at least now i can confirm 
2.6.8 and above, but i think all 2.6), if the onlink tip does not work, you 
can compile a 2.6 kernel with random enabled and try again. You can post the 
script that you are ussing, it should convine PREROUTING -j MARK rules with 
ip ru add fw MARK table mytableX. 
 
--
Luciano

On Wednesday 16 August 2006 13:03, Luciano Ruete wrote:
> On Wednesday 16 August 2006 10:11, Jacques Rompen wrote:
> > On 8/14/06, Luciano Ruete <luciano@lugmen.org.ar> wrote:
> > > On Monday 14 August 2006 12:42, Jacques Rompen wrote:
> > > > Ok ive been trying to get this to work for about half a year
now. Ive
> > > > searched all over the internet for a solution for
> > > > my  problem. Ive found some solutions, but they only led me
to yet
> > > > more problems.
> > > >
> > > > What we want to do is the following:
> > > > I live in a student complex with 7 other people. Every room
has its
> > > > own internet connection from the same ISP.
> > > > Ip, gateway, subnet are asigned through dhcp on mac-adres
basis.
> > > > Every internet connection is capped at 20mbit
> > > > up/down.
> > > > We want to get all computers on an internal network. So we
need some
> > >
> > > sort
> > >
> > > > of router that accepts all 8 internet
> > > > connections and routes it out over a gbit nic -> switch
-> internal
> > >
> > > network
> > >
> > > > So we have a computer with 10 nics and "hopefully"
enough internal
> > > > bandwith.
> > >
> > > one simple solution is in this post[1] (read all the thread if
you
> > > whant a context). Bassically it uses iptables CONNMARK to
remembre to
> > > which iface(and
> > > certanly wich src ip) each connection flow belongs.
> > >
> > > You will have to use same gw in all ifaces, maybe the ip route
"onlink"
> > > ption(command line) make your script simpler. No need to use
julian
> > > patch and
> > > certanly DO NOT USE CONFIG_IP_ROUTE_MULTIPATH_CACHED in your
kernel
> > > config.
> > >
> > > Feel free to post any doubts.
> > >
> > > Greets.
> > >
> > > [1]http://mailman.ds9a.nl/pipermail/lartc/2006q2/018964.html
> > > --
> > > --
> > > Luciano
> >
> > Hi Luciano,
> >
> > Ive tried the above solution but then with juliano''s patch
instead of
> > conmark. It didnt work then because this part of the script:
> >
> > ip ro add default  nexthop via x.x.x.x dev eth1 weight 1 nexthop via
> > y.y.y.y dev eth2
> > ip route add default table provider1 via x.x.x.x dev eth1
> > ip route add default table provider2 via y.y.y.y dev eth2
> > # and most important
> > ip rule add fwmark 0x1 table provider1
> > ip rule add fwmark 0x2 table provider2
> >
> > somehow looked at the default gateway x.x.x.x or y.y.y.y instead of to
> > the "dev ethx"
> > Because the gateway adresses were both the same x.x.x.x = y.y.y.y all
the
> > outgoing packets were routed out over eth1.
>
> ok, what if you try with onlink:
> ip ro add default  nexthop via x.x.x.x eth1 onlink nexthop via x.x.x.x eth2
> onlink and so on...
and forget to say, to be really shure, del any trace of
x.x.x.x dev ethx scope link 
in "ip ro ls". Where x.x.x.x is the gw ip, the onlink will force to
drop the
packet on the named device even if there is no physical route to that host.
--
Luciano

On 16-08-2006 16:18, Jacques Rompen wrote:
> 
> 
> On 8/16/06, *Jarek Poplawski* <jarkao2@o2.pl
<mailto:jarkao2@o2.pl>> wrote:
> 
>     As far as I know multipath cached should work for routing output
>     of a current box but it''s not working for router (if something
>     has not changed lately). Multipath without "cached" should do
>     simple randomization, but maybe for the begining it would be
>     better to assign fixed paths by source ip and save experimenting
>     with multipaths for later.
> 
>     Jarek P.
> 
>  
> Hi Jarek,
>  
> I will try with only multipath.
>  
> Ive tried the static routes. But still ive had trouble getting this code 
> to work
> ip ro add default  nexthop via x.x.x.x dev eth1 weight 1 nexthop via
> y.y.y.y dev eth2
> ip route add default table provider1 via x.x.x.x dev eth1
> ip route add default table provider2 via y.y.y.y dev eth2
> because x.x.x.x = y.y.y.y in my case. Somehow it doesnt look at the eth 
> device anymore.
Anyway in my opinion it should work.
Try something like this:

#IP_ROUTE_MULTIPATH = "y"
#(if 2.6.x  IP_ROUTE_MULTIPATH_CACHED = "n" !)
#Here with CONNMARK method (so without Anastasov''s patch!).
#Change 192.168.0.0/24 and eth0 to your local network
#address/mask and interface.

ip route add default nexthop via x.x.x.x dev eth1 \
   nexthop via x.x.x.x dev eth2

ip route add 192.168.0.0/24 dev eth0 table 101
ip route add default via x.x.x.x dev eth1 table 101

ip route add 192.168.0.0/24 dev eth0 table 102
ip route add default via x.x.x.x dev eth2 table 102

ip rule add fwmark 1 table 101
ip rule add fwmark 2 table 102

iptables -t mangle -A POSTROUTING -o eth1 -j MARK --set-mark 1
iptables -t mangle -A POSTROUTING -o eth2 -j MARK --set-mark 2

iptables -t mangle -A POSTROUTING -j CONNMARK --save-mark
iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark

echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter

You should also add NAT (MASQERADE) rules.

Nexthop devices will be changed randomly per flow (with some 
caching) so try with destinations from different networks.

Jarek P.

On Thursday 17 August 2006 03:54, Jarek Poplawski wrote:
> On 16-08-2006 16:18, Jacques Rompen wrote:
> > On 8/16/06, *Jarek Poplawski* <jarkao2@o2.pl
<mailto:jarkao2@o2.pl>>
> > wrote:
> >
> >     As far as I know multipath cached should work for routing output
> >     of a current box but it''s not working for router (if
something
> >     has not changed lately). Multipath without "cached"
should do
> >     simple randomization, but maybe for the begining it would be
> >     better to assign fixed paths by source ip and save experimenting
> >     with multipaths for later.
> >
> >     Jarek P.
> >
> >
> > Hi Jarek,
> >
> > I will try with only multipath.
> >
> > Ive tried the static routes. But still ive had trouble getting this
code
> > to work
> > ip ro add default  nexthop via x.x.x.x dev eth1 weight 1 nexthop via
> > y.y.y.y dev eth2
> > ip route add default table provider1 via x.x.x.x dev eth1
> > ip route add default table provider2 via y.y.y.y dev eth2
> > because x.x.x.x = y.y.y.y in my case. Somehow it doesnt look at the
eth
> > device anymore.
>
> Anyway in my opinion it should work.
> Try something like this:
>
> #IP_ROUTE_MULTIPATH = "y"
> #(if 2.6.x  IP_ROUTE_MULTIPATH_CACHED = "n" !)
> #Here with CONNMARK method (so without Anastasov''s patch!).
> #Change 192.168.0.0/24 and eth0 to your local network
> #address/mask and interface.
>
> ip route add default nexthop via x.x.x.x dev eth1 \
>    nexthop via x.x.x.x dev eth2
>
> ip route add 192.168.0.0/24 dev eth0 table 101
> ip route add default via x.x.x.x dev eth1 table 101
>
> ip route add 192.168.0.0/24 dev eth0 table 102
> ip route add default via x.x.x.x dev eth2 table 102
>
> ip rule add fwmark 1 table 101
> ip rule add fwmark 2 table 102
>
> iptables -t mangle -A POSTROUTING -o eth1 -j MARK --set-mark 1
> iptables -t mangle -A POSTROUTING -o eth2 -j MARK --set-mark 2
>
> iptables -t mangle -A POSTROUTING -j CONNMARK --save-mark
> iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark
>
> echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter

if all ethx are on the same switch, you will need:
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
#you can also do it in a per iface basis

by default, if you recive an arp request(whohas) on a iface(iface1), asking 
for an ip that another iface(iface2) has, then the arp request is ansered 
with the MAC address of the first iface(iface1). If all devices are in the 
same fisical segment this can lead to a single device answer for all IPs, or 
to a total caos. 

This one is very hard to catch and i forgot to mention, now solved :-)

--
Luciano

On 8/17/06, Luciano Ruete <luciano@lugmen.org.ar>
wrote:
>
>
>
> if all ethx are on the same switch, you will need:
> echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
> #you can also do it in a per iface basis
>
> by default, if you recive an arp request(whohas) on a iface(iface1),
> asking
> for an ip that another iface(iface2) has, then the arp request is ansered
> with the MAC address of the first iface(iface1). If all devices are in the
> same fisical segment this can lead to a single device answer for all IPs,
> or
> to a total caos.
>
> This one is very hard to catch and i forgot to mention, now solved :-)
>
> --
> Luciano

I will try all this tommorow, don''t have acces to the box right now.

About turning of arp. If you turn them all of, wouldnt the nics (ip''s)
be
unfindable from the outside world? Or does the switch they connect to
respond to such an arp request aswell?
But what you are saying is that if i had a box with 2 nics each connected to
a different ISP, so each nic with a different gateway and ip. That if nic1
would recieve a arp request for the ip from nic2, it would respond with
mac-adres from nic1. So that could mean that all packets would come in over
nic1, even if they have destination ip(nic2)? Seems to me that this is never
realy desirable. Especialy if nic2 would have an static ip and i would
unplug nic2 on purpose.


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

On Thursday 17 August 2006 14:48, Jacques Rompen wrote:
> On 8/17/06, Luciano Ruete <luciano@lugmen.org.ar> wrote:
> > if all ethx are on the same switch, you will need:
> > echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
> > #you can also do it in a per iface basis
> >
> > by default, if you recive an arp request(whohas) on a iface(iface1),
> > asking
> > for an ip that another iface(iface2) has, then the arp request is
ansered
> > with the MAC address of the first iface(iface1). If all devices are in
> > the same fisical segment this can lead to a single device answer for
all
> > IPs, or
> > to a total caos.
> >
> > This one is very hard to catch and i forgot to mention, now solved :-)
> >
> > --
> > Luciano
>
> I will try all this tommorow, don''t have acces to the box right
now.
>
> About turning of arp. If you turn them all of, wouldnt the nics
(ip''s) be
> unfindable from the outside world? Or does the switch they connect to
> respond to such an arp request aswell?
what you''re turning off is that the device answers arp(whohas) requests
for
ips that are not from that specific device itself. The ips assigned to the 
device will cotinue answer as normal.
> But what you are saying is that if i had a box with 2 nics each connected
> to a different ISP, so each nic with a different gateway and ip. That if
> nic1 would recieve a arp request for the ip from nic2, it would respond
> with mac-adres from nic1. 
exactly!
> So that could mean that all packets would come in 
> over nic1, even if they have destination ip(nic2)? 
exactly, and will arrive destiny anyway, but they are incoming for the wrong 
iface.
> Seems to me that this is 
> never realy desirable.  
It is a really cuestionable default, but for shure that there are reasons for 
this(maybe a thread in lkml archives could answer this cuestion), reasons 
that i do not know.
> Especialy if nic2 would have an static ip and i  
> would unplug nic2 on purpose.
well thats the other scenario where i face the same problem, first guess is 
obvious MAC cache, and you tend to think that is fucked up, and no, it is 
this crossed arp answer.

--
Luciano

Hi !

Luciano: thanks for your posts on lugmen.org.ar and lartc!

I''ve read all of them, and I''ve a question:

What diferrence bewtween this:

-> ip ro add default  NEXTHOP via x.x.x.x dev eth1 weight 1  \ 
			    NEXTHOP via y.y.y.y dev eth2


and this:

->  ip route add x.x.x.x MPATH RR via y.y.y.y via z.z.z.z

I work with 2.4 Kernel Series. Is connvenient upgrade to 2.6 ?

Other question about this:

iptables -t mangle -A POSTROUTING -o eth0 -j CONNMARK --set-mark 0x1
iptables -t mangle -A POSTROUTING -o eth1 -j CONNMARK --set-mark 0x2

My problem is that I use CONNMARK''s marks in order to mark P2P traffic.
Can I''ve a way to have diferents mark with CONNMARK ?

Thank you!!

Andres.

On 8/17/06, Luciano Ruete <luciano@lugmen.org.ar>
wrote:
>
> >
> > I will try all this tommorow, don''t have acces to the box
right now.
> >
> > About turning of arp. If you turn them all of, wouldnt the nics
(ip''s)
> be
> > unfindable from the outside world? Or does the switch they connect to
> > respond to such an arp request aswell?
>
> what you''re turning off is that the device answers arp(whohas)
requests
> for
> ips that are not from that specific device itself. The ips assigned to the
> device will cotinue answer as normal.
>
> > But what you are saying is that if i had a box with 2 nics each
> connected
> > to a different ISP, so each nic with a different gateway and ip. That
if
> > nic1 would recieve a arp request for the ip from nic2, it would
respond
> > with mac-adres from nic1.
>
> exactly!
>
> > So that could mean that all packets would come in
> > over nic1, even if they have destination ip(nic2)?
>
> exactly, and will arrive destiny anyway, but they are incoming for the
> wrong
> iface.
>
> > Seems to me that this is
> > never realy desirable.
>
> It is a really cuestionable default, but for shure that there are reasons
> for
> this(maybe a thread in lkml archives could answer this cuestion), reasons
> that i do not know.
>
> > Especialy if nic2 would have an static ip and i
> > would unplug nic2 on purpose.
>
> well thats the other scenario where i face the same problem, first guess
> is
> obvious MAC cache, and you tend to think that is fucked up, and no, it is
> this crossed arp answer.
>
> --
> Luciano

Hi Luciano and Jarek,

Thanks for all your help so far.

Ive had some try on the box this weekend. Unfortunatly its not in the
student complex anymore, but it will be very soon. So for now ive simulated
the situation by connecting 2 nics to a switch and that switch to a router.
This is the script i used so far, no NAT rules yet because i have no extra
computer to connect to the box. Its Jareks script completed with some hints
from Luciano


ip route add default nexthop via x.x.x.x dev eth1 onlink\
   nexthop via x.x.x.x dev eth2 onlink

ip route add 192.168.0.0/24 dev eth0 table 101
ip route add default via x.x.x.x dev eth1 table 101

ip route add 192.168.0.0/24 dev eth0 table 102
ip route add default via x.x.x.x dev eth2 table 102

ip rule add fwmark 1 table 101
ip rule add fwmark 2 table 102

iptables -t mangle -A POSTROUTING -o eth1 -j MARK --set-mark 1
iptables -t mangle -A POSTROUTING -o eth2 -j MARK --set-mark 2

iptables -t mangle -A POSTROUTING -j CONNMARK --save-mark
iptables -t mangle -A PREROUTING -j CONNMARK --restore-mark

echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter

echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore

I am thinking if it wouldn''t be better to include a src = ipe1 to the
"route
add default" entries.
ofcourse this would go wrong if i would recive a new ip from the isp on one
of the nics. So if its not needed then i would rather leave it out.
ive tested the script with

ip route get from IPE1 to 204.152.189.113
ip route get from IPE2 to 204.152.189.113

it does indeed answer wit eth1 for the first and eth2 for the seccond. It
didnt do this with my other attempts. Im not sure if this is a real vallid
tes though.

Regards,

Jacques


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc