thr3ads.net - LARTC - Multi-path routing only using last nexthop in default route. [Jan 2006]

If this information is useful, please help other people find it:
Share via:

Jody Shumaker

2006-Jan-16 22:27 UTC

Multi-path routing only using last nexthop in default route.

I''ve applied julian''s paches to a 2.6.14 gentoo kernel with
the appropiate
options enabled, and i''m using a modified version of the mpath2.sh
script
also available on julian''s site http://www.ssi.bg/~ja/

Overall everything works nearly perfectly.  Incomming connections to either
the fios (PPPoE) connection, or cable modem get routed back out correctly.
The problem I''m having is with the default gateway.

${IP} route add default table 221 proto static \
            nexthop via ${EXTGW2} dev ${EXTIF2} weight 1\
            nexthop via ${EXTGW1} dev ${EXTIF1} weight 5

With this command, connections going out from my network always seem to use
the GW1 route.  I if I reverse the order of the nexthop''s to list GW1
first
and GW2 second, then the reverse happens and all outgoign connections use
GW2.  I''m going to attempt to test this better by attempting a large
number
of connections to a large list of ip''s, but in running this setup
I''ve never
seen it use both gateway''s.

Possible symptom I''ve noticed is that in the route cache, there tends
to be
2 entries, with the used gateway showing up first, such as:

83.53.46.36 from 192.168.0.128 via 10.9.44.15 dev ppp0  src 192.168.0.1
    cache <src-direct>  mtu 1492 advmss 1452 metric10 64 iif eth0
83.53.46.36 from 192.168.0.128 via 66.189.76.1 dev eth1  src 192.168.0.1
    cache <src-direct>  mtu 1500 advmss 1460 metric10 64 iif eth0

In this case, ppp0 was listed as the second nexthop.  If I reversed the
order of the nexthop''s, then the order they show up in the cache
reverses.
Does anyone have any ideas why both gateways don''t seem to be used?
Please
let me know if any other information about my setup would be useful.
I''ve
been experimenting with various other options like turning off rp_filter for
both connections, adjusting weight''s, performing the connections from
various internal ip''s to various external ip''s to insure new
route lookups,
etc. I''ve just never ever seen it use anything but the last specified
nexthop which has lead me to fear it''s a problem with the patches
and/or my
kernel.

Thanks,
Jody


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Jody Shumaker

2006-Jan-17 01:59 UTC

head link

Re: Multi-path routing only using last nexthop in default route.

I found that for ppp devices, i should ony define the next  hop with the
dev, not a via.  However this still didn''t fix my problem, but
I''ve narrowed
down my problem a little further.

# ip route get 66.189.123.136
66.189.123.136 dev ppp0  src 71.248.183.244
    cache  mtu 1492 advmss 1452 metric10 64
# ip route get 66.189.123.137
66.189.123.137 dev ppp0  src 66.189.76.198
    cache  mtu 1492 advmss 1452 metric10 64

It does properly do a 5:1 round robin choice , but only the src changes, not
the dev.  The above I believe should really have outputted for the second
route:
66.189.123.137 dev eth1  src 66.189.76.198
    cache  mtu 1492 advmss 1452 metric10 64

I''m not sure what is wrong with my config, as I''ve gone over
and over it. My
best guess is that something is wrong in the kernel I compiled with the
patches.

# ip rule show
0:      from all lookup local
50:     from all lookup main
201:    from 71.248.183.244 lookup 201
202:    from 66.189.76.198/22 lookup 202
221:    from all lookup 221
32766:  from all lookup main
32767:  from all lookup default

# ip route show table main
10.9.44.15 dev ppp0  proto kernel  scope link  src 71.248.183.244
192.168.100.0/24 dev eth1  proto kernel  scope link  src 192.168.100.2
192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.1
66.189.76.0/22 dev eth1  proto kernel  scope link  src 66.189.76.198
127.0.0.0/8 dev lo  scope link

# ip route show table 201
default via 10.9.44.15 dev ppp0  proto static  src 71.248.183.244
prohibit default  proto static  metric 1

# ip route show table 202
default via 66.189.76.1 dev eth1  proto static  src 66.189.76.198
prohibit default  proto static  metric 1

# ip route show table 221
default  proto static
        nexthop via 66.189.76.1  dev eth1 weight 1
        nexthop dev ppp0 weight 5


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Alexander Samad

2006-Jan-17 04:16 UTC

head link

Re: Re: Multi-path routing only using last nexthop in default route.

On Mon, Jan 16, 2006 at 08:59:32PM -0500, Jody Shumaker
wrote:> I found that for ppp devices, i should ony define the next  hop with the
> dev, not a via.  However this still didn''t fix my problem, but
I''ve narrowed
> down my problem a little further.
> 
> # ip route get 66.189.123.136
> 66.189.123.136 dev ppp0  src 71.248.183.244
>     cache  mtu 1492 advmss 1452 metric10 64
> # ip route get 66.189.123.137
> 66.189.123.137 dev ppp0  src 66.189.76.198
>     cache  mtu 1492 advmss 1452 metric10 64
doesnt the second ip r g just show you what you have in the route cache,
when I try it on my multi home machine

default  metric 5 
        nexthop via 141.168.16.1  dev eth0 weight 3
        nexthop via 220.233.1.45  dev ppp0 weight 4

but this might be because I don''t have the round-robin patch applied to
the kernel.

> 
> It does properly do a 5:1 round robin choice , but only the src changes,
not
> the dev.  The above I believe should really have outputted for the second
> route:
> 66.189.123.137 dev eth1  src 66.189.76.198
>     cache  mtu 1492 advmss 1452 metric10 64
> 
> I''m not sure what is wrong with my config, as I''ve gone
over and over it. My
> best guess is that something is wrong in the kernel I compiled with the
> patches.
> 
> # ip rule show
> 0:      from all lookup local
> 50:     from all lookup main
> 201:    from 71.248.183.244 lookup 201
> 202:    from 66.189.76.198/22 lookup 202
> 221:    from all lookup 221
> 32766:  from all lookup main
> 32767:  from all lookup default
> 
> # ip route show table main
> 10.9.44.15 dev ppp0  proto kernel  scope link  src 71.248.183.244
> 192.168.100.0/24 dev eth1  proto kernel  scope link  src 192.168.100.2
> 192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.1
> 66.189.76.0/22 dev eth1  proto kernel  scope link  src 66.189.76.198
> 127.0.0.0/8 dev lo  scope link
> 
> # ip route show table 201
> default via 10.9.44.15 dev ppp0  proto static  src 71.248.183.244
> prohibit default  proto static  metric 1
> 
> # ip route show table 202
> default via 66.189.76.1 dev eth1  proto static  src 66.189.76.198
> prohibit default  proto static  metric 1
> 
> # ip route show table 221
> default  proto static
>         nexthop via 66.189.76.1  dev eth1 weight 1
>         nexthop dev ppp0 weight 5
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Jody Shumaker

2006-Jan-17 05:37 UTC

head link

Re: Re: Multi-path routing only using last nexthop in default route.

Yes, it just shows you what is in the cache, but I was specifying ip
addresses that weren''t in the cache yet. I also tried doing traceroutes
from
an internal pc, and those always ended up going over the 1 interface.
I''ve
also tried adjusting the weights to 1:1 and opening up numerous connections
to multiple ftp''s.

Also for comparison, if I change the order of the nexthop''s
I''ll instead get
effectively the reverse.

# ip route get 66.1.1.11
66.1.1.11 via 66.189.76.1 dev eth1  src 71.248.183.63
    cache  mtu 1500 advmss 1460 metric10 64
# ip route get 66.1.1.12
66.1.1.12 via 66.189.76.1 dev eth1  src 66.189.76.198
    cache  mtu 1500 advmss 1460 metric10 64

It always is pointing to dev eth1 while with the reverse order it was ppp0.
All this by only changing the order of the nexthops. I went through and
double checked that I did apply julian''s patches to the kernel source I
last
built with.

- Jody

On 1/16/06, Alexander Samad <alex@samad.com.au>
wrote:>
> On Mon, Jan 16, 2006 at 08:59:32PM -0500, Jody Shumaker wrote:
> > I found that for ppp devices, i should ony define the next  hop with
the
> > dev, not a via.  However this still didn''t fix my problem,
but I''ve
> narrowed
> > down my problem a little further.
> >
> > # ip route get 66.189.123.136
> > 66.189.123.136 dev ppp0  src 71.248.183.244
> >     cache  mtu 1492 advmss 1452 metric10 64
> > # ip route get 66.189.123.137
> > 66.189.123.137 dev ppp0  src 66.189.76.198
> >     cache  mtu 1492 advmss 1452 metric10 64
>
> doesnt the second ip r g just show you what you have in the route cache,
> when I try it on my multi home machine
>
> default  metric 5
>         nexthop via 141.168.16.1  dev eth0 weight 3
>         nexthop via 220.233.1.45  dev ppp0 weight 4
>
> but this might be because I don''t have the round-robin patch
applied to
> the kernel.
>
>
> >
> > It does properly do a 5:1 round robin choice , but only the src
changes,
> not
> > the dev.  The above I believe should really have outputted for the
> second
> > route:
> > 66.189.123.137 dev eth1  src 66.189.76.198
> >     cache  mtu 1492 advmss 1452 metric10 64
> >
> > I''m not sure what is wrong with my config, as I''ve
gone over and over
> it. My
> > best guess is that something is wrong in the kernel I compiled with
the
> > patches.
> >
> > # ip rule show
> > 0:      from all lookup local
> > 50:     from all lookup main
> > 201:    from 71.248.183.244 lookup 201
> > 202:    from 66.189.76.198/22 lookup 202
> > 221:    from all lookup 221
> > 32766:  from all lookup main
> > 32767:  from all lookup default
> >
> > # ip route show table main
> > 10.9.44.15 dev ppp0  proto kernel  scope link  src 71.248.183.244
> > 192.168.100.0/24 dev eth1  proto kernel  scope link  src 192.168.100.2
> > 192.168.0.0/24 dev eth0  proto kernel  scope link  src 192.168.0.1
> > 66.189.76.0/22 dev eth1  proto kernel  scope link  src 66.189.76.198
> > 127.0.0.0/8 dev lo  scope link
> >
> > # ip route show table 201
> > default via 10.9.44.15 dev ppp0  proto static  src 71.248.183.244
> > prohibit default  proto static  metric 1
> >
> > # ip route show table 202
> > default via 66.189.76.1 dev eth1  proto static  src 66.189.76.198
> > prohibit default  proto static  metric 1
> >
> > # ip route show table 221
> > default  proto static
> >         nexthop via 66.189.76.1  dev eth1 weight 1
> >         nexthop dev ppp0 weight 5
>
> > _______________________________________________
> > LARTC mailing list
> > LARTC@mailman.ds9a.nl
> > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
>
> iD8DBQFDzG+WkZz88chpJ2MRAkQJAKDaR/QeqheUntdS2pX/j5IMWoQ5FQCeLX4V
> EHKOXCpr481+FEt8h5bRzDo> =ukY3
> -----END PGP SIGNATURE-----
>
>
>

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Alexander Samad

2006-Jan-17 19:23 UTC

head link

Re: Re: Multi-path routing only using last nexthop in default route.

On Tue, Jan 17, 2006 at 12:37:48AM -0500, Jody Shumaker
wrote:> Yes, it just shows you what is in the cache, but I was specifying ip
> addresses that weren''t in the cache yet. I also tried doing
traceroutes from
> an internal pc, and those always ended up going over the 1 interface.
I''ve
> also tried adjusting the weights to 1:1 and opening up numerous connections
> to multiple ftp''s.
> 
> Also for comparison, if I change the order of the nexthop''s
I''ll instead get
> effectively the reverse.
> 
> # ip route get 66.1.1.11
> 66.1.1.11 via 66.189.76.1 dev eth1  src 71.248.183.63
>     cache  mtu 1500 advmss 1460 metric10 64
> # ip route get 66.1.1.12
> 66.1.1.12 via 66.189.76.1 dev eth1  src 66.189.76.198
>     cache  mtu 1500 advmss 1460 metric10 64
your right I tried it on my machine 
for x in $(seq 1 10); do ip r g 1.1.1.$x; done
1.1.1.1 via 220.233.1.45 dev ppp0  src 220.233.15.63 
    cache  mtu 1492 advmss 1452 metric 10 64
1.1.1.2 via 220.233.1.45 dev ppp0  src 141.168.16.16 
    cache  mtu 1492 advmss 1452 metric 10 64
1.1.1.3 via 220.233.1.45 dev ppp0  src 220.233.15.63 
    cache  mtu 1492 advmss 1452 metric 10 64
1.1.1.4 via 220.233.1.45 dev ppp0  src 141.168.16.16 
    cache  mtu 1492 advmss 1452 metric 10 64
1.1.1.5 via 220.233.1.45 dev ppp0  src 220.233.15.63 
    cache  mtu 1492 advmss 1452 metric 10 64
1.1.1.6 via 220.233.1.45 dev ppp0  src 220.233.15.63 
    cache  mtu 1492 advmss 1452 metric 10 64
1.1.1.7 via 220.233.1.45 dev ppp0  src 220.233.15.63 
    cache  mtu 1492 advmss 1452 metric 10 64
1.1.1.8 via 220.233.1.45 dev ppp0  src 220.233.15.63 
    cache  mtu 1492 advmss 1452 metric 10 64
1.1.1.9 via 220.233.1.45 dev ppp0  src 220.233.15.63 
    cache  mtu 1492 advmss 1452 metric 10 64
1.1.1.10 via 220.233.1.45 dev ppp0  src 220.233.15.63 
    cache  mtu 1492 advmss 1452 metric 10 64

just the src address is changing, I am pretty sure this used work at
some point in time, i am using 2.6.14-1-smp, iptables v1.3.3
> 
> It always is pointing to dev eth1 while with the reverse order it was ppp0.
> All this by only changing the order of the nexthops. I went through and
> double checked that I did apply julian''s patches to the kernel
source I last
> built with.
> 
> - Jody
> 
> On 1/16/06, Alexander Samad <alex@samad.com.au> wrote:
> >
> > On Mon, Jan 16, 2006 at 08:59:32PM -0500, Jody Shumaker wrote:
> > > I found that for ppp devices, i should ony define the next  hop
with the
> > > dev, not a via.  However this still didn''t fix my
problem, but I''ve
> > narrowed
> > > down my problem a little further.
> > >
> > > # ip route get 66.189.123.136
> > > 66.189.123.136 dev ppp0  src 71.248.183.244
> > >     cache  mtu 1492 advmss 1452 metric10 64
> > > # ip route get 66.189.123.137
> > > 66.189.123.137 dev ppp0  src 66.189.76.198
> > >     cache  mtu 1492 advmss 1452 metric10 64
> >
> > doesnt the second ip r g just show you what you have in the route
cache,
> > when I try it on my multi home machine
> >
> > default  metric 5
> >         nexthop via 141.168.16.1  dev eth0 weight 3
> >         nexthop via 220.233.1.45  dev ppp0 weight 4
> >
> > but this might be because I don''t have the round-robin patch
applied to
> > the kernel.
> >
> >
> > >
> > > It does properly do a 5:1 round robin choice , but only the src
changes,
> > not
> > > the dev.  The above I believe should really have outputted for
the
> > second
> > > route:
> > > 66.189.123.137 dev eth1  src 66.189.76.198
> > >     cache  mtu 1492 advmss 1452 metric10 64
> > >
> > > I''m not sure what is wrong with my config, as
I''ve gone over and over
> > it. My
> > > best guess is that something is wrong in the kernel I compiled
with the
> > > patches.
> > >
> > > # ip rule show
> > > 0:      from all lookup local
> > > 50:     from all lookup main
> > > 201:    from 71.248.183.244 lookup 201
> > > 202:    from 66.189.76.198/22 lookup 202
> > > 221:    from all lookup 221
> > > 32766:  from all lookup main
> > > 32767:  from all lookup default
> > >
> > > # ip route show table main
> > > 10.9.44.15 dev ppp0  proto kernel  scope link  src 71.248.183.244
> > > 192.168.100.0/24 dev eth1  proto kernel  scope link  src
192.168.100.2
> > > 192.168.0.0/24 dev eth0  proto kernel  scope link  src
192.168.0.1
> > > 66.189.76.0/22 dev eth1  proto kernel  scope link  src
66.189.76.198
> > > 127.0.0.0/8 dev lo  scope link
> > >
> > > # ip route show table 201
> > > default via 10.9.44.15 dev ppp0  proto static  src 71.248.183.244
> > > prohibit default  proto static  metric 1
> > >
> > > # ip route show table 202
> > > default via 66.189.76.1 dev eth1  proto static  src 66.189.76.198
> > > prohibit default  proto static  metric 1
> > >
> > > # ip route show table 221
> > > default  proto static
> > >         nexthop via 66.189.76.1  dev eth1 weight 1
> > >         nexthop dev ppp0 weight 5
> >
> > > _______________________________________________
> > > LARTC mailing list
> > > LARTC@mailman.ds9a.nl
> > > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
> >
> >
> >
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.4.2 (GNU/Linux)
> >
> > iD8DBQFDzG+WkZz88chpJ2MRAkQJAKDaR/QeqheUntdS2pX/j5IMWoQ5FQCeLX4V
> > EHKOXCpr481+FEt8h5bRzDo> > =ukY3
> > -----END PGP SIGNATURE-----
> >
> >
> >
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Jody Shumaker

2006-Jan-17 21:53 UTC

head link

Re: Re: Multi-path routing only using last nexthop in default route.

Does anyone have a confirmed to be working multipath setup? I''d like to
see
their route output and confirm that this really is an issue.  The issue
might actually be something else and this output is expected? I''m just
sticking on this because the order of nexthops is what changes the behavior,
which seems wrong.

Also, if I try retieving paths from an internal address to an external, it
will always use only the last nexthop.

# for x in $(seq 1 10); do ip route get 66.1.1.$x from 192.168.0.128 iif
eth0; done
66.1.1.1 from 192.168.0.128 dev ppp0  src 192.168.0.1
    cache <src-direct>  mtu 1492 advmss 1452 metric10 64 iif eth0
66.1.1.2 from 192.168.0.128 dev ppp0  src 192.168.0.1
    cache <src-direct>  mtu 1492 advmss 1452 metric10 64 iif eth0
etc.

I''m using 2.6.14-gentoo-r5 #4 SMP PREEMPT w/ julian''s patches
and iptables
v1.3.4

- Jody

On 1/17/06, Alexander Samad <alex@samad.com.au>
wrote:>
> On Tue, Jan 17, 2006 at 12:37:48AM -0500, Jody Shumaker wrote:
> > Yes, it just shows you what is in the cache, but I was specifying ip
> > addresses that weren''t in the cache yet. I also tried doing
traceroutes
> from
> > an internal pc, and those always ended up going over the 1 interface.
> I''ve
> > also tried adjusting the weights to 1:1 and opening up numerous
> connections
> > to multiple ftp''s.
> >
> > Also for comparison, if I change the order of the nexthop''s
I''ll instead
> get
> > effectively the reverse.
> >
> > # ip route get 66.1.1.11
> > 66.1.1.11 via 66.189.76.1 dev eth1  src 71.248.183.63
> >     cache  mtu 1500 advmss 1460 metric10 64
> > # ip route get 66.1.1.12
> > 66.1.1.12 via 66.189.76.1 dev eth1  src 66.189.76.198
> >     cache  mtu 1500 advmss 1460 metric10 64
>
> your right I tried it on my machine
> for x in $(seq 1 10); do ip r g 1.1.1.$x; done
> 1.1.1.1 via 220.233.1.45 dev ppp0  src 220.233.15.63
>     cache  mtu 1492 advmss 1452 metric 10 64
> 1.1.1.2 via 220.233.1.45 dev ppp0  src 141.168.16.16
>     cache  mtu 1492 advmss 1452 metric 10 64
> 1.1.1.3 via 220.233.1.45 dev ppp0  src 220.233.15.63
>     cache  mtu 1492 advmss 1452 metric 10 64
> 1.1.1.4 via 220.233.1.45 dev ppp0  src 141.168.16.16
>     cache  mtu 1492 advmss 1452 metric 10 64
> 1.1.1.5 via 220.233.1.45 dev ppp0  src 220.233.15.63
>     cache  mtu 1492 advmss 1452 metric 10 64
> 1.1.1.6 via 220.233.1.45 dev ppp0  src 220.233.15.63
>     cache  mtu 1492 advmss 1452 metric 10 64
> 1.1.1.7 via 220.233.1.45 dev ppp0  src 220.233.15.63
>     cache  mtu 1492 advmss 1452 metric 10 64
> 1.1.1.8 via 220.233.1.45 dev ppp0  src 220.233.15.63
>     cache  mtu 1492 advmss 1452 metric 10 64
> 1.1.1.9 via 220.233.1.45 dev ppp0  src 220.233.15.63
>     cache  mtu 1492 advmss 1452 metric 10 64
> 1.1.1.10 via 220.233.1.45 dev ppp0  src 220.233.15.63
>     cache  mtu 1492 advmss 1452 metric 10 64
>
> just the src address is changing, I am pretty sure this used work at
> some point in time, i am using 2.6.14-1-smp, iptables v1.3.3
>
>
>

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Alexander Samad

2006-Jan-18 02:13 UTC

head link

Re: Re: Multi-path routing only using last nexthop in default route.

On Tue, Jan 17, 2006 at 04:53:06PM -0500, Jody Shumaker
wrote:> Does anyone have a confirmed to be working multipath setup? I''d
like to see
> their route output and confirm that this really is an issue.  The issue
> might actually be something else and this output is expected? I''m
just
> sticking on this because the order of nexthops is what changes the
behavior,
> which seems wrong.
I think mine is working, because I se traffic heading out of the second
interface (ones that I know have originated from my box), plus when I check the
cache table there are entries for both
interfaces.

just can''t prove it right now 8(

A
> 
> Also, if I try retieving paths from an internal address to an external, it
> will always use only the last nexthop.
> 
> # for x in $(seq 1 10); do ip route get 66.1.1.$x from 192.168.0.128 iif
> eth0; done
> 66.1.1.1 from 192.168.0.128 dev ppp0  src 192.168.0.1
>     cache <src-direct>  mtu 1492 advmss 1452 metric10 64 iif eth0
> 66.1.1.2 from 192.168.0.128 dev ppp0  src 192.168.0.1
>     cache <src-direct>  mtu 1492 advmss 1452 metric10 64 iif eth0
> etc.
> 
> I''m using 2.6.14-gentoo-r5 #4 SMP PREEMPT w/ julian''s
patches and iptables
> v1.3.4
> 
> - Jody
> 
> On 1/17/06, Alexander Samad <alex@samad.com.au> wrote:
> >
> > On Tue, Jan 17, 2006 at 12:37:48AM -0500, Jody Shumaker wrote:
> > > Yes, it just shows you what is in the cache, but I was specifying
ip
> > > addresses that weren''t in the cache yet. I also tried
doing traceroutes
> > from
> > > an internal pc, and those always ended up going over the 1
interface.
> > I''ve
> > > also tried adjusting the weights to 1:1 and opening up numerous
> > connections
> > > to multiple ftp''s.
> > >
> > > Also for comparison, if I change the order of the
nexthop''s I''ll instead
> > get
> > > effectively the reverse.
> > >
> > > # ip route get 66.1.1.11
> > > 66.1.1.11 via 66.189.76.1 dev eth1  src 71.248.183.63
> > >     cache  mtu 1500 advmss 1460 metric10 64
> > > # ip route get 66.1.1.12
> > > 66.1.1.12 via 66.189.76.1 dev eth1  src 66.189.76.198
> > >     cache  mtu 1500 advmss 1460 metric10 64
> >
> > your right I tried it on my machine
> > for x in $(seq 1 10); do ip r g 1.1.1.$x; done
> > 1.1.1.1 via 220.233.1.45 dev ppp0  src 220.233.15.63
> >     cache  mtu 1492 advmss 1452 metric 10 64
> > 1.1.1.2 via 220.233.1.45 dev ppp0  src 141.168.16.16
> >     cache  mtu 1492 advmss 1452 metric 10 64
> > 1.1.1.3 via 220.233.1.45 dev ppp0  src 220.233.15.63
> >     cache  mtu 1492 advmss 1452 metric 10 64
> > 1.1.1.4 via 220.233.1.45 dev ppp0  src 141.168.16.16
> >     cache  mtu 1492 advmss 1452 metric 10 64
> > 1.1.1.5 via 220.233.1.45 dev ppp0  src 220.233.15.63
> >     cache  mtu 1492 advmss 1452 metric 10 64
> > 1.1.1.6 via 220.233.1.45 dev ppp0  src 220.233.15.63
> >     cache  mtu 1492 advmss 1452 metric 10 64
> > 1.1.1.7 via 220.233.1.45 dev ppp0  src 220.233.15.63
> >     cache  mtu 1492 advmss 1452 metric 10 64
> > 1.1.1.8 via 220.233.1.45 dev ppp0  src 220.233.15.63
> >     cache  mtu 1492 advmss 1452 metric 10 64
> > 1.1.1.9 via 220.233.1.45 dev ppp0  src 220.233.15.63
> >     cache  mtu 1492 advmss 1452 metric 10 64
> > 1.1.1.10 via 220.233.1.45 dev ppp0  src 220.233.15.63
> >     cache  mtu 1492 advmss 1452 metric 10 64
> >
> > just the src address is changing, I am pretty sure this used work at
> > some point in time, i am using 2.6.14-1-smp, iptables v1.3.3
> >
> >
> >
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Ciprian Constantinescu

2006-Jan-18 02:22 UTC

head link

Re: Re: Multi-path routing only using last nexthop in default route.

It works. I have a Debian and the tests were the following:

   1. multiple traceroute to multiple hosts. you can observe the gateway
   that changes
   2. i run a squid server and i entered http://whatismyip.com multiple
   times from the same computer in the lan. the ip changed between the 2
   providers i have
   3. i run mrtg on the box, so the graph said it all


On 1/18/06, Alexander Samad <alex@samad.com.au>
wrote:>
> On Tue, Jan 17, 2006 at 04:53:06PM -0500, Jody Shumaker wrote:
> > Does anyone have a confirmed to be working multipath setup?
I''d like to
> see
> > their route output and confirm that this really is an issue.  The
issue
> > might actually be something else and this output is expected?
I''m just
> > sticking on this because the order of nexthops is what changes the
> behavior,
> > which seems wrong.
>
> I think mine is working, because I se traffic heading out of the second
> interface (ones that I know have originated from my box), plus when I
> check the cache table there are entries for both
> interfaces.
>
> just can''t prove it right now 8(
>
> A
>
> >
> > Also, if I try retieving paths from an internal address to an
external,
> it
> > will always use only the last nexthop.
> >
> > # for x in $(seq 1 10); do ip route get 66.1.1.$x from 192.168.0.128
iif
> > eth0; done
> > 66.1.1.1 from 192.168.0.128 dev ppp0  src 192.168.0.1
> >     cache <src-direct>  mtu 1492 advmss 1452 metric10 64 iif
eth0
> > 66.1.1.2 from 192.168.0.128 dev ppp0  src 192.168.0.1
> >     cache <src-direct>  mtu 1492 advmss 1452 metric10 64 iif
eth0
> > etc.
> >
> > I''m using 2.6.14-gentoo-r5 #4 SMP PREEMPT w/
julian''s patches and
> iptables
> > v1.3.4
> >
> > - Jody
> >
> > On 1/17/06, Alexander Samad <alex@samad.com.au> wrote:
> > >
> > > On Tue, Jan 17, 2006 at 12:37:48AM -0500, Jody Shumaker wrote:
> > > > Yes, it just shows you what is in the cache, but I was
specifying ip
> > > > addresses that weren''t in the cache yet. I also
tried doing
> traceroutes
> > > from
> > > > an internal pc, and those always ended up going over the 1
> interface.
> > > I''ve
> > > > also tried adjusting the weights to 1:1 and opening up
numerous
> > > connections
> > > > to multiple ftp''s.
> > > >
> > > > Also for comparison, if I change the order of the
nexthop''s I''ll
> instead
> > > get
> > > > effectively the reverse.
> > > >
> > > > # ip route get 66.1.1.11
> > > > 66.1.1.11 via 66.189.76.1 dev eth1  src 71.248.183.63
> > > >     cache  mtu 1500 advmss 1460 metric10 64
> > > > # ip route get 66.1.1.12
> > > > 66.1.1.12 via 66.189.76.1 dev eth1  src 66.189.76.198
> > > >     cache  mtu 1500 advmss 1460 metric10 64
> > >
> > > your right I tried it on my machine
> > > for x in $(seq 1 10); do ip r g 1.1.1.$x; done
> > > 1.1.1.1 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > 1.1.1.2 via 220.233.1.45 dev ppp0  src 141.168.16.16
> > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > 1.1.1.3 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > 1.1.1.4 via 220.233.1.45 dev ppp0  src 141.168.16.16
> > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > 1.1.1.5 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > 1.1.1.6 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > 1.1.1.7 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > 1.1.1.8 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > 1.1.1.9 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > 1.1.1.10 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > >     cache  mtu 1492 advmss 1452 metric 10 64
> > >
> > > just the src address is changing, I am pretty sure this used work
at
> > > some point in time, i am using 2.6.14-1-smp, iptables v1.3.3
> > >
> > >
> > >
>
> > _______________________________________________
> > LARTC mailing list
> > LARTC@mailman.ds9a.nl
> > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
>
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.2 (GNU/Linux)
>
> iD8DBQFDzaQ5kZz88chpJ2MRArpVAKDVe8ET7m4Qz09HhxbykV93/meFtACg3bWT
> GgOZ8WrUWiAmIT83rrRCRR8> =7U0w
> -----END PGP SIGNATURE-----
>
>
> _______________________________________________
> LARTC mailing list
> LARTC@mailman.ds9a.nl
> http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
>
>
>

--
Ciprian Constantinescu
mobile: +40745192289
e-mail: c_ciprian_ro@yahoo.com
e-mail: c.ciprian@gmail.com
yahoo messenger: c_ciprian_ro@yahoo.com
msn messenger: c_ciprian_ro@yahoo.com


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Jody Shumaker

2006-Jan-18 04:59 UTC

head link

Re: Re: Multi-path routing only using last nexthop in default route.

I understand it can work, my point was I want to figure out what is
different or wrong with my setup compared to one where it is working.  Would
you mind posting the results of
ip rule show
ip route show table all

what kernel version you have, and possibly the output from this command?

for x in $(seq 1 10); do ip r g 1.1.1.$x; done

I''m kind of grasping at straws because everything I''ve tried
short of
debugging/hacking the kernel code hasn''t worked.

Thanks,
Jody


On 1/17/06, Ciprian Constantinescu <c.ciprian@gmail.com>
wrote:>
> It works. I have a Debian and the tests were the following:
>
>    1. multiple traceroute to multiple hosts. you can observe the
>    gateway that changes
>    2. i run a squid server and i entered http://whatismyip.com multiple
>    times from the same computer in the lan. the ip changed between the 2
>    providers i have
>    3. i run mrtg on the box, so the graph said it all
>
>
> On 1/18/06, Alexander Samad <alex@samad.com.au> wrote:
>
> > On Tue, Jan 17, 2006 at 04:53:06PM -0500, Jody Shumaker wrote:
> > > Does anyone have a confirmed to be working multipath setup?
I''d like
> > to see
> > > their route output and confirm that this really is an issue.  The
> > issue
> > > might actually be something else and this output is expected?
I''m just
> > > sticking on this because the order of nexthops is what changes
the
> > behavior,
> > > which seems wrong.
> >
> > I think mine is working, because I se traffic heading out of the
second
> > interface (ones that I know have originated from my box), plus when I
> > check the cache table there are entries for both
> > interfaces.
> >
> > just can''t prove it right now 8(
> >
> > A
> >
> > >
> > > Also, if I try retieving paths from an internal address to an
> > external, it
> > > will always use only the last nexthop.
> > >
> > > # for x in $(seq 1 10); do ip route get 66.1.1.$x from
192.168.0.128iif
> > > eth0; done
> > > 66.1.1.1 from 192.168.0.128 dev ppp0  src 192.168.0.1
> > >     cache <src-direct>  mtu 1492 advmss 1452 metric10 64
iif eth0
> > > 66.1.1.2 from 192.168.0.128 dev ppp0  src 192.168.0.1
> > >     cache <src-direct>  mtu 1492 advmss 1452 metric10 64
iif eth0
> > > etc.
> > >
> > > I''m using 2.6.14-gentoo-r5 #4 SMP PREEMPT w/
julian''s patches and
> > iptables
> > > v1.3.4
> > >
> > > - Jody
> > >
> > > On 1/17/06, Alexander Samad <alex@samad.com.au > wrote:
> > > >
> > > > On Tue, Jan 17, 2006 at 12:37:48AM -0500, Jody Shumaker
wrote:
> > > > > Yes, it just shows you what is in the cache, but I was
specifying
> > ip
> > > > > addresses that weren''t in the cache yet. I
also tried doing
> > traceroutes
> > > > from
> > > > > an internal pc, and those always ended up going over
the 1
> > interface.
> > > > I''ve
> > > > > also tried adjusting the weights to 1:1 and opening up
numerous
> > > > connections
> > > > > to multiple ftp''s.
> > > > >
> > > > > Also for comparison, if I change the order of the
nexthop''s I''ll
> > instead
> > > > get
> > > > > effectively the reverse.
> > > > >
> > > > > # ip route get 66.1.1.11
> > > > > 66.1.1.11 via 66.189.76.1 dev eth1  src 71.248.183.63
> > > > >     cache  mtu 1500 advmss 1460 metric10 64
> > > > > # ip route get 66.1.1.12
> > > > > 66.1.1.12 via 66.189.76.1 dev eth1  src 66.189.76.198
> > > > >     cache  mtu 1500 advmss 1460 metric10 64
> > > >
> > > > your right I tried it on my machine
> > > > for x in $(seq 1 10); do ip r g 1.1.1.$x; done
> > > > 1.1.1.1 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > > 1.1.1.2 via 220.233.1.45 dev ppp0  src 141.168.16.16
> > > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > > 1.1.1.3 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > > 1.1.1.4 via 220.233.1.45 dev ppp0  src 141.168.16.16
> > > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > > 1.1.1.5 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > > 1.1.1.6 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > > 1.1.1.7 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > > 1.1.1.8 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > > 1.1.1.9 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > > 1.1.1.10 via 220.233.1.45 dev ppp0  src 220.233.15.63
> > > >     cache  mtu 1492 advmss 1452 metric 10 64
> > > >
> > > > just the src address is changing, I am pretty sure this used
work at
> > > > some point in time, i am using 2.6.14-1-smp, iptables v1.3.3
> > > >
> > > >
> > > >
> >
> > > _______________________________________________
> > > LARTC mailing list
> > > LARTC@mailman.ds9a.nl
> > > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
> >
> >
> >
> > -----BEGIN PGP SIGNATURE-----
> > Version: GnuPG v1.4.2 (GNU/Linux)
> >
> > iD8DBQFDzaQ5kZz88chpJ2MRArpVAKDVe8ET7m4Qz09HhxbykV93/meFtACg3bWT
> > GgOZ8WrUWiAmIT83rrRCRR8> > =7U0w
> > -----END PGP SIGNATURE-----
> >
> >
> > _______________________________________________
> > LARTC mailing list
> > LARTC@mailman.ds9a.nl
> > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
> >
> >
> >
>
>
> --
> Ciprian Constantinescu
> mobile: +40745192289
> e-mail: c_ciprian_ro@yahoo.com
> e-mail: c.ciprian@gmail.com
> yahoo messenger: c_ciprian_ro@yahoo.com
> msn messenger: c_ciprian_ro@yahoo.com

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Julian Anastasov

2006-Jan-18 08:54 UTC

head link

Re: Multi-path routing only using last nexthop in default route.

Hello,

On Mon, 16 Jan 2006, Jody Shumaker wrote:
> ${IP} route add default table 221 proto static \
>             nexthop via ${EXTGW2} dev ${EXTIF2} weight 1\
>             nexthop via ${EXTGW1} dev ${EXTIF1} weight 5
>
> With this command, connections going out from my network always seem to use
> the GW1 route.  I if I reverse the order of the nexthop''s to list
GW1 first
> and GW2 second, then the reverse happens and all outgoign connections use
> GW2.  I''m going to attempt to test this better by attempting a
large number
> of connections to a large list of ip''s, but in running this setup
I''ve never
> seen it use both gateway''s.
	Do you have script to ping/arping the gateways on eth device(s)?
The NOARP devices are always preferred if the GWs on ARP devices are
not marked reachable in ARP cache.

Regards

--
Julian Anastasov <ja@ssi.bg>

Jody Shumaker

2006-Jan-20 00:54 UTC

head link

Re: Multi-path routing only using last nexthop in default route.

Further information, I''ve found that if I run with multiple nexthops
for the
default route, eventually I get a kernel panic. I can of course speed up the
process by having some script repeatedly retrieve new routes.  Some of the
info on screen when it happens:
EFLAGS: 00010202
EIP is at _stext+0x3feffd68/0x28
Stack:
rcu_do_batch+0x1f/0x70
rcu_process_callbacks+0x5d/0x70
tasklet_action+0x73/0xd0
__do_softirq+0xc5/0xe0
do_softirq+0x32/0x40
irq_exit+0x3e/0x40
do_IRQ+0x1e/0x30
common_interrupt+0x1a/0x20
acpi_processor_idle+0x129/0x2b4
cpu_idle+0x67/0x70
start_kernel+0x161/0x180
unknown_bootoption+0x0/0x1b0

Both times it happened, the stack trace was pretty much the same. If I
instead only specify one nexthop, then I don''t get this kernel panic.

for x in $(seq 1 254); do for y in $(seq 1 254); do for z in $(seq 1 254);
do ip r g 1.$x.$y.$z; done; done; done
If I run that, with 2 nexthop''s, it kernel panics fairly quickly, after
500-1000 iterations roughly?  With only 1 nexthop specified it had gone up
to roughly 4000 iterations and beyond with no kernel panic.  Possibly the
patches aren''t compatible with other patches the gentoo kernel has
applied... going to research what I can on that.

- Jody

On 1/18/06, Jody Shumaker <jody.shumaker@gmail.com>
wrote:>
>
>         Do you have script to ping/arping the gateways on eth device(s)?
> > The NOARP devices are always preferred if the GWs on ARP devices are
> > not marked reachable in ARP cache.
>
>
> Yes, I have a script that pings the gateways on both devices every minute.
> I also just checked with `arp -an` and the gateway for the eth1 device is
> listed.  Also, if I swap the order of the nexthop''s then I can
have it favor
> the eth1 over ppp0 always.
>
> - Jody
>

_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Maybe Matching Threads

Search for more apparently analagous threads

LARTC - Jan 2006 - Multi-path routing only using last nexthop in default route.

Multi-path routing only using last nexthop in default route.

Re: Multi-path routing only using last nexthop in default route.

Re: Re: Multi-path routing only using last nexthop in default route.

Re: Re: Multi-path routing only using last nexthop in default route.

Re: Re: Multi-path routing only using last nexthop in default route.

Re: Re: Multi-path routing only using last nexthop in default route.

Re: Re: Multi-path routing only using last nexthop in default route.

Re: Re: Multi-path routing only using last nexthop in default route.

Re: Re: Multi-path routing only using last nexthop in default route.

Re: Multi-path routing only using last nexthop in default route.

Re: Multi-path routing only using last nexthop in default route.

Maybe Matching Threads