I am running 2.4.20 with Julian Anastasov''s patches
(routes-2.4.20-9.diff), iptables v1.2.6a, iproute2-ss010824 on Debian.
I have set up our internal gateway to multihome 2 T1''s as described in
http://www.linuxvirtualserver.org/~julian/nano.txt . The only
difference is that I use multiple IP''s on the external interfaces.
EXA A.B.C.225
-------------------- A.B.C.226-230/27 eth0
GATEWAY ----------------- INT
192.168.0.1/24 eth1
-------------------- J.K.L.98-102/27 eth2
EXB J.K.L.97
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
link/ether 00:07:e9:cb:64:81 brd ff:ff:ff:ff:ff:ff
inet A.B.C.226/27 brd A.B.C.255 scope global eth0
inet A.B.C.227/27 brd A.B.C.255 scope global secondary eth0:0
inet A.B.C.228/27 brd A.B.C.255 scope global secondary eth0:1
inet A.B.C.229/27 brd A.B.C.255 scope global secondary eth0:2
inet A.B.C.230/27 brd A.B.C.255 scope global secondary eth0:3
3: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
link/ether 00:07:e9:cb:64:82 brd ff:ff:ff:ff:ff:ff
inet 192.168.0.1/24 brd 192.168.0.255 scope global eth1
4: eth2: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
link/ether 00:04:75:aa:c7:eb brd ff:ff:ff:ff:ff:ff
inet J.K.L.98/27 brd J.K.L.127 scope global eth2
inet J.K.L.99/27 brd J.K.L.127 scope global secondary eth2:0
inet J.K.L.100/27 brd J.K.L.127 scope global secondary eth2:1
inet J.K.L.101/27 brd J.K.L.127 scope global secondary eth2:2
inet J.K.L.102/27 brd J.K.L.127 scope global secondary eth2:3
I use the following to set up routing and NAT (relevant excerpts shown).
EXA_IF=eth0
EXA_NET=`ip addr ls $EXA_IF | awk ''/inet/ { print $2 }'' | head
-1`
EXA_IP=${EXA_NET//\/*/}
EXA_GW=A.B.C.225
EXA_RNG=A.B.C.230
EXB_IF=eth2
EXB_NET=`ip addr ls $EXB_IF | awk ''/inet/ { print $2 }'' | head
-1`
EXB_IP=${EXB_NET//\/*/}
EXB_GW=J.K.L.97
EXB_RNG=J.K.L.102
INT_IF=eth1
INT_NET=`ip addr ls $INT_IF | awk ''/inet/ { print $2 }'' | head
-1`
INT_IP=${INT_NET//\/*/}
ip rule add prio 50 table main
ip route del default table main 2>/dev/null
ip rule add priority 100 from $EXB_NET table EXB
ip route add default via $EXB_GW dev $EXB_IF src $EXB_IP \
proto static table EXB
ip route append prohibit default table EXB metric 1 proto static
ip rule add priority 150 from $EXA_NET table EXA
ip route add default via $EXA_GW dev $EXA_IF src $EXA_IP \
proto static table EXA
ip route append prohibit default table EXA metric 1 proto static
# set up default route as multipath route
ip rule add priority 200 table multi
ip route add default table multi proto static \
nexthop via $EXB_GW dev $EXB_IF weight 1 \
nexthop via $EXA_GW dev $EXA_IF weight 1
ip route flush cache
iptables -A FORWARD -m state \
--state ESTABLISHED,RELATED -j ACCEPT
iptables -A INPUT -m state \
--state ESTABLISHED,RELATED -j ACCEPT
iptables -A OUTPUT -m state \
--state ESTABLISHED,RELATED -j ACCEPT
iptables -t nat -A PREROUTING -m state \
--state ESTABLISHED,RELATED -j ACCEPT
iptables -t nat -A POSTROUTING -m state \
--state ESTABLISHED,RELATED -j ACCEPT
iptables -t nat -A OUTPUT -m state \
--state ESTABLISHED,RELATED -j ACCEPT
iptables -t nat -A POSTROUTING -s $INT_NET -o $EXB_IF \
-j SNAT --to-source $EXB_IP-$EXB_RNG
iptables -t nat -A POSTROUTING -s $INT_NET -o $EXA_IF \
-j SNAT --to-source $EXA_IP-$EXA_RNG
( there is more but it is not relevant )
Here''s the problem:
Packets get lost along one of the routes. Everything runs smoothly when
the EXB route is selected, but some packets get dropped when the EXA
route is selected.
For example:
If I ping a host outside our network and the EXB route is
chosen, I get the replies. On a bridge machine between the
internal gateway and the EXB provider I can see that iptables is
cycling through the range of SNAT ip''s.
If I ping a host outside out network and the EXA route is
chosen, I get the replies sometimes and sometimes not. On the
bridge machine I can see the packets go back and forth but they
get discarded when they return to the internal gateway. I log my
iptables drops and they don''t show up anywhere.
Inspection shows that the packets get lost when certain of the EXA
range of ip''s is chosen. It works when the external ip chosen is .226,
.227, and .230 only. When the SNAT picks .228 or .229 it fails.
Also, 2 identical outgoing routes appear in ''ip ro ls cache'',
but only
one return route is created. When the EXB route is chosen I only see
one outgoing and one incoming route.
Any ideas? I''ve made sure that there are no other iptables rules to
drop those packets. Arp looks fine.
I can fix the problem by limiting the SNAT on EXA to one IP address and
not using a range. When I do that everything works like it is supposed
to. I''d like to be able to use a range of ip''s though.
