Hello, LARTC mailing readers, I hope u can help with this mysterious
issue
i'm having with my linux box acting as a router.
Scenario:
Linux running 2.6.8.1 /w julians patches
Latest iproute (iproute2-ss040702)
4 NICS
-----------------
|
x eth0 (63.43.x.x) network mask (255.255.240.0)
|
|
x eth1 (63.43.x.x) network mask (255.255.240.0)
|
|
Linux x eth2 (63.43.x.x) network mask (255.255.240.0)
|
|
x eth3 (172.31.0.6) LAN of routers
| (172.31.0.1, 172.31.0.2, 172.31.0.3)
| network mask (255.255.255.248)
x eth4 (172.31.0.254) LAN of users
| network mask (255.255.255.0)
|
-----------------
FIRST OF ALL:
I know it has not a sane routing table in relation with eth3 and eth4,
but it work
because in the main table first appears eth3 entry, and the range of the
lan
is (172.31.0.8 - 172.31.0.254).
As you can deduce I have the same provider for eth0, eth1, eth2, it's a
cable operator.
Im using iptables / ip for doing port based routing. So I can separate
well known services
in my enviroment.
This is basically my running config:
xerver:~# ip ru ls
0: from all lookup local
32758: from all fwmark 0x8 lookup balanceo2
32759: from all fwmark 0x7 lookup cablemodem3
32760: from all fwmark 0x5 lookup cablemodem
32761: from all fwmark 0x6 lookup cablemodem2
32762: from all fwmark 0x4 lookup balanceo
32763: from all fwmark 0x3 lookup router3
32764: from all fwmark 0x2 lookup router2
32765: from all fwmark 0x1 lookup router1
32766: from all lookup main
32767: from all lookup default
xerver:~# cat /etc/iproute2/rt_tables
255 local
254 main
253 default
0 unspec
200 balanceo2
201 balanceo
202 cablemodem
204 router1
205 router2
206 router3
207 cablemodem2
208 cablemodem3
Routes declarations:
ip route add default via 172.31.0.1 dev eth3 src 172.31.0.6 table router1
ip route add default via 172.31.0.2 dev eth3 src 172.31.0.6 table router2
ip route add default via 172.31.0.3 dev eth3 src 172.31.0.6 table router3
ip route add default proto static via 63.43.1.1 dev eth0 src $IP1 table
cablemodem
ip route add default proto static via 63.43.1.1 dev eth1 src $IP2 table
cablemodem2
ìp route add default proto static via 63.43.1.1 dev eth2 src $IP3 table
cablemodem3
ip route add table balanceo2 default proto static \
nexthop via $GW dev eth0 weight 1 \
nexthop via $GW2 dev eth1 weight 1 \
nexthop via $GW3 dev eth2 weight 1
(also tested with eql modifier, but no success)
I've read eql only load-balances per packet basis, on my network, that's
counterproducent. ?¿
(I need per connection basis load-balance)
xerver:~# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use
Iface
172.31.0.0 0.0.0.0 255.255.255.248 U 0 0 0 eth3
172.31.0.0 0.0.0.0 255.255.255.0 U 0 0 0 eth4
63.43.x.x 0.0.0.0 255.255.224.0 U 0 0 0 eth0
63.43.x.x 0.0.0.0 255.255.224.0 U 0 0 0 eth1
63.43.x.x 0.0.0.0 255.255.224.0 U 0 0 0 eth2
0.0.0.0 63.43.1.1 0.0.0.0 UG 0 0 0 eth0
Ok, every cablemodem has the same gateway, they all are in the same
subnet.
So eth0 has gw 63.43.1.1, eth1 has gw 63.43.1.1 and eth2 has gw
63.43.1.1.
Those three lines are 1Mbit Down 512 Up
Ok, when I mark my packets to follow a specific route through a specific
gw, it works
great. I have to say I had some trouble with arp flux problem, but, I
suppose if I've
been able to make the three links download at the same time it's correct
to suppose
that the arp flux is corrected.
Tkweaks to /proc
echo "0" > /proc/sys/net/ipv4/conf/all/rp_filter
echo "0" > /proc/sys/net/ipv4/conf/eth0/rp_filter
echo "0" > /proc/sys/net/ipv4/conf/eth1/rp_filter
echo "0" > /proc/sys/net/ipv4/conf/eth2/rp_filter
echo "0" > /proc/sys/net/ipv4/conf/eth3/rp_filter
echo "0" > /proc/sys/net/ipv4/conf/eth4/rp_filter
echo "1" > /proc/sys/net/ipv4/conf/eth0/hidden
echo "1" > /proc/sys/net/ipv4/conf/eth1/hidden
echo "1" > /proc/sys/net/ipv4/conf/eth2/hidden
echo "1" > /proc/sys/net/ipv4/conf/eth3/hidden
echo "1" > /proc/sys/net/ipv4/conf/eth0/arp_filter
echo "1" > /proc/sys/net/ipv4/conf/eth1/arp_filter
echo "1" > /proc/sys/net/ipv4/conf/eth2/arp_filter
echo "1" > /proc/sys/net/ipv4/conf/eth3/arp_filter
My troubles comes with load balance outgoing connections, I use the
table balanceo2 for
this purpose, and looks like this:
xerver:~#ip ro sh table balanceo2
default proto static
nexthop via 62.43.0.1 dev eth0 weight 1
nexthop via 62.43.0.1 dev eth1 weight 1
nexthop via 62.43.0.1 dev eth2 weight 1
But when I mark www traffic like this:
iptables -A PREROUTING -t mangle -i eth4 -p tcp --dport 80 -j MARK
--set-mark 8
Everything breaks down, no www traffic is possible for the clients.
I've tried the same with ICMP echo requests and it seemed to work.
I've tkweaked a bit /proc for route cache times expiration but no
success.
I don't know what I'm missing.
NOTE: Some time ago, my ISP had another network structure, and
sometimes, I got
different gw in each cablemodem, and load balance worked fine.
I think the problem comes in the routing cache, or expiration times,
really I
don't know.
I would appreciate any help.
Thanks in advance.
Xisco Fernandez.