Hello,
I have following problem with multiple gateways.
Question is: why e-mail traffic goes throught eth2? I want to pass all
e-mail traffic throught eth0 (10.48.32.1) without multiple connections
to the same destination through ONLY ONE gateway. Is it possible to
switch off load balancing(??) between gateways? I want it to be static.
Is there any way to save and restore settings from ip tool like iptables
does? (iptables-save, restore)
Here We go (addresses changed because of security):
# ip addr
1: lo: <LOOPBACK,UP> mtu 16436 qdisc noqueue
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 brd 127.255.255.255 scope host lo
2: eth0: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 100
    link/ether 00:04:ac:d9:7f:7c brd ff:ff:ff:ff:ff:ff
    inet 10.48.32.10/16 brd 10.48.255.255 scope global eth0
    inet 10.48.32.3/24 brd 10.48.32.255 scope global eth0:1
    inet x.x.156.102/29 brd x.x.156.103 scope global eth0:2
4: eth2: <BROADCAST,UP> mtu 1500 qdisc pfifo_fast qlen 150
    link/ether 00:c0:f0:4d:e2:13 brd ff:ff:ff:ff:ff:ff
    inet x.x.153.158/30 brd x.x.153.159 scope global eth2
# ip rule show
0:      from all lookup local
32761:  from x.x.153.158 lookup formus2
32762:  from 192.168.100.0/24 lookup formus2
32763:  from 10.48.32.10 fwmark        4 lookup formus2
32764:  from 0.0.0.0 fwmark        2 lookup rose
32765:  from 10.48.64.2 lookup formus2
32766:  from all lookup main
32767:  from all lookup default
# iptables -L -n -v -x -t mangle
Chain PREROUTING (policy ACCEPT 3738441 packets, 2204536651 bytes)
    pkts      bytes target     prot opt in     out  source
destination
93282 68889253 MARK       all  --  *      * 0.0.0.0/0
x.x.153.158      MARK set 0x5
141     6309 MARK       tcp  --  *      * ! x.x.153.158
x.x.153.158      tcp dpt:2000 MARK set 0x3
21289  5294990 MARK       tcp  --  eth0   * 0.0.0.0/0
10.48.32.10        tcp dpt:8080 MARK set 0x4
8239  1870997 TOS        tcp  --  *      * 0.0.0.0/0
0.0.0.0/0          tcp spt:22 TOS set 0x10
101040  7491165 TOS        tcp  --  *      * 0.0.0.0/0
0.0.0.0/0          tcp dpt:22 TOS set 0x10
Chain OUTPUT (policy ACCEPT 981349 packets, 622327299 bytes)
    pkts      bytes target     prot opt in     outq
source               destination
   13918  1479670 MARK       tcp  --  *      *       10.48.32.10
!10.0.0.0/8         tcp dpt:80 MARK set 0x4
   12612 15769675 MARK       tcp  --  *      *       10.48.32.10
!10.0.0.0/8         tcp dpt:25 MARK set 0x2
   18605  3526169 MARK       all  --  *      *       10.48.64.2
!10.0.0.0/8         MARK set 0x4
   12584  1208166 TOS        tcp  --  *      *
10.48.32.10          0.0.0.0/0          tcp dpt:80 TOS set 0x08
# ip route list tab formus2
10.0.0.0/8 via 10.48.32.1 dev eth0  proto kernel
default via x.x.153.157 dev eth2
# ip route list tab rose
default via 10.48.32.1 dev eth0
# ip route list tab main
10.48.32.6 dev ppp0  proto kernel  scope link  src 10.48.32.10
10.48.32.1 via 10.48.32.10 dev eth0
x.x.153.156/30 dev eth2  proto kernel  scope link  src x.x.153.158
x.x.156.96/29 dev eth0  proto kernel  scope link  src x.x.156.102
192.168.100.0/24 dev eth1  proto kernel  scope link  src 192.168.100.1
10.48.32.0/24 dev eth0  proto kernel  scope link  src 10.48.32.3
10.48.0.0/16 dev eth0  proto kernel  scope link  src 10.48.32.10
10.0.0.0/8 via 10.48.32.1 dev eth0
127.0.0.0/8 dev lo  scope link
default
        nexthop via 10.48.32.1  dev eth0 weight 1
        nexthop via x.x.153.157  dev eth2 weight 2
# mailq -vs
smtp/nnn.com.pl:
        R/96581-30410: (2 tries, expires in 4d23h) smtp; 500 (connect to
www.nnn.com.pl [x.x.150.125|25|10.48.32.10|35339]:
Connection timed out)
# route -Cn|grep x.x.150.125
10.48.32.10     x.x.150.125 10.48.32.1            0      0        0 eth0
10.48.32.10     x.x.150.125 x.x.153.157         0      0        5 eth2
10.48.32.10     x.x.150.125 x.x.153.157         0      0        0 eth2
(one destination routed throught two interfaces?)
# iptables -L -n -t nat
Chain PREROUTING (policy ACCEPT)
target     prot opt source               destination
DNAT       all  -- !10.48.0.0/24         10.48.32.3 to:192.168.100.1
Chain POSTROUTING (policy ACCEPT)
target     prot opt source               destination
SNAT       all  --  0.0.0.0/0            0.0.0.0/0          MARK match
0x4 to:62.32.153.158
SNAT       all  --  10.48.64.2          !10.0.0.0/8         to:
x.x.153.158
SNAT       all  --  192.168.100.0/24    !192.168.100.0/24 to:x.x.153.158
Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
My networks:
10/8    - intranet
10.48.64.2/32    - me (I want to be routed through eth2 except e-mail
traffic)
others - routed normally through eth0 except squid connections.
I wrote the following startup script:
#!/bin/bash
IP=''/sbin/ip''
SRCHOST=''10.48.64.2''
D1L=''x.x.153.158''
D1R=''x.x.153.157''
D2L=''10.48.32.10''       # local out if
D2R=''10.48.32.1''        # remote out if
HIT=''192.168.100.1''     # Hitachi
echo "0" >/proc/sys/net/ipv4/conf/all/rp_filter
$IP link set eth2 dynamic on multicast off txqueuelen 150
$IP route del default via 10.48.32.1 dev eth0
$IP route add default via $D1R dev eth2 table formus2 proto kernel
$IP route add default via $D2R dev eth0 table rose proto kernel
$IP route add to 10/8 via $D2R dev eth0
#$IP route add to $D2R/32 dev eth0 via $D2L
$IP rule add from $SRCHOST table formus2
$IP rule add from all table rose fwmark 02                      # use
rose if mark is 2
$IP rule add from $D2L table formus2 fwmark 04              # Squid from
10.48.32.10 out by eth2
$IP rule add from 192.168.100.0/24 table formus2
$IP rule add from $D1L table formus2
$IP rule add from $D2L table rose
$IP route add proto kernel default nexthop via 10.48.32.1 weight 1 dev
eth0 nexthop via x.x.153.157 weight 2 dev eth2
$IP route add table formus2 from 10.48.64.0/24 to 10/8 via 10.48.32.1
dev eth0 proto kernel
$IP route flush cache
# ip -V
ip utility, iproute2-ss010824
Further thanks,
Wojtek Sobola
Unix System Engineer
S&T Poland