thr3ads.net - LARTC - Conntrack, nat and multipath

If this information is useful, please help other people find it:
Share via:

Erik S. Johansen

2006-Apr-09 18:42 UTC

Conntrack, nat and multipath - what is wrong here?

I have a gentoo 2.6.14 box with 4 nics, LAN/DMZ/PUB1/PUB2

LAN and DMZ have a 1918 /22 each, PUB1 and PUB2 have a /29 each of which 5 ips 
are assigned.

Using the mangle table, I give all packets a mark (according to local 
policies) in the range 1-10. Using ip rule, i pass marks 1-5 through the pub1 
route table, and marks 6-10 through the pub2 routing table. Using the nat 
table, I SNAT to one of the 10 IPs assigned from the two /29''s.


1) Now, if i remove the default route (via PUB1 gw) from the main table, 
everything halts. Why? 


2) If I pass a forwarded tcp syn packet out on the PUB2 interface, with the 
correct SNAT ip, I can see the syn+ack returning from the external server. 
Logging then indicates that this packet gets passed through 
mangle/PREROUTING, after which it appears to simply be lost. It''s
definitely
not going out on any of the 4 NICs. This contrasts with packets being passed 
out on PUB1, where everything works fine, conntrack recognizes the syn+ack 
and the reply gets correctly forwarded to the LAN box i''m using to
test. It
*seems* like conntrack simply is not able to match the incoming syn+ack with 
the outgoing syn. BUT, if i try to connect to the dsl router on PUB2, 
everything''s fine. I suspect i got something very wrong with my routing
rules/tables, but I can''t figure out what.



Addresses shown are sanitized, 1.1.1.136/29 is PUB1, 2.2.2.116/29 is PUB2, 
3.3.3.* is the external server i''ve been testing against.

eth0: LAN
eth1: DMZ
eth2: PUB2
eth3: PUB1




eos ~ # ip rule show
0:      from all lookup local
30000:  from all fwmark 0x1 lookup pub1
30000:  from all fwmark 0x2 lookup pub1
30000:  from all fwmark 0x3 lookup pub1
30000:  from all fwmark 0x4 lookup pub1
30000:  from all fwmark 0x5 lookup pub1
30000:  from all fwmark 0x6 lookup pub2
30000:  from all fwmark 0x7 lookup pub2
30000:  from all fwmark 0x8 lookup pub2
30000:  from all fwmark 0x9 lookup pub2
30000:  from all fwmark 0xa lookup pub2
31000:  from 1.1.1.139 lookup pub1
31000:  from 1.1.1.140 lookup pub1
31000:  from 1.1.1.141 lookup pub1
31000:  from 1.1.1.142 lookup pub1
31000:  from 1.1.1.137 lookup pub1
31000:  from 2.2.2.218 lookup pub2
31000:  from 2.2.2.219 lookup pub2
31000:  from 2.2.2.220 lookup pub2
31000:  from 2.2.2.221 lookup pub2
31000:  from 2.2.2.222 lookup pub2
33000:  from all lookup main

eos ~ # ip route show table pub1
1.1.1.136/29 dev eth3  scope link  src 1.1.1.139
2.2.2.216/29 dev eth2  scope link  src 2.2.2.218
192.168.4.0/22 dev eth1  scope link  src 192.168.4.1
192.168.0.0/22 dev eth0  scope link  src 192.168.0.1
127.0.0.0/8 dev lo  scope link
default via 1.1.1.138 dev eth3

eos ~ # ip route show table pub2
1.1.1.136/29 dev eth3  scope link  src 1.1.1.139
2.2.2.216/29 dev eth2  scope link  src 2.2.2.218
192.168.4.0/22 dev eth1  scope link  src 192.168.4.1
192.168.0.0/22 dev eth0  scope link  src 192.168.0.1
127.0.0.0/8 dev lo  scope link
default via 2.2.2.217 dev eth2

eos ~ # ip route show table main
1.1.1.136/29 dev eth3  proto kernel  scope link  src 1.1.1.139
2.2.2.216/29 dev eth2  proto kernel  scope link  src 2.2.2.218
192.168.4.0/22 dev eth1  proto kernel  scope link  src 192.168.4.1
192.168.0.0/22 dev eth0  proto kernel  scope link  src 192.168.0.1
127.0.0.0/8 dev lo  scope link
default via 1.1.1.138 dev eth3

eos ~ # iptables -t filter -nvL
Chain INPUT (policy ACCEPT 5314 packets, 2615K bytes)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `filter/INPUT:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `filter/INPUT:''

Chain FORWARD (policy ACCEPT 184K packets, 162M bytes)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `filter/FORWARD:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `filter/FORWARD:''

Chain OUTPUT (policy ACCEPT 2261 packets, 277K bytes)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `filter/OUTPUT:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `filter/OUTPUT:''

eos ~ # iptables -t mangle -nvL
Chain PREROUTING (policy ACCEPT 188K packets, 165M bytes)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/PREROUTING:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/PREROUTING:''
    2   468 MARK14     all  --  *      *       0.0.0.0/0            
192.168.4.0/22      state NEW
 2903 2444K MARK13     all  --  *      *       0.0.0.0/0            
192.168.0.0/22      state NEW
   60  6098 MARK12     all  --  *      *       0.0.0.0/0            
1.1.1.136/29     state NEW
 1692  136K MARK11     all  --  *      *       0.0.0.0/0            
2.2.2.216/29   state NEW
    0     0 MARK6      tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 state NEW
  109  5232 MARK6      tcp  --  eth0   *       0.0.0.0/0            0.0.0.0/0
tcp dpt:119 state NEW
   54  2592 MARK6      tcp  --  eth1   *       0.0.0.0/0            0.0.0.0/0
tcp dpt:119 state NEW
    0     0 MARK2      all  --  *      *       192.168.1.20         
213.239.111.0/29    state NEW
 3223  243K MARK10     all  --  eth1   *       0.0.0.0/0            0.0.0.0/0
state NEW
 1054 66052 MARK1      all  --  eth0   *       0.0.0.0/0            0.0.0.0/0
state NEW

Chain INPUT (policy ACCEPT 5409 packets, 2648K bytes)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/INPUT:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/INPUT:''

Chain FORWARD (policy ACCEPT 188K packets, 165M bytes)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/FORWARD:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/FORWARD:''

Chain OUTPUT (policy ACCEPT 2302 packets, 283K bytes)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/OUTPUT:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/OUTPUT:''

Chain POSTROUTING (policy ACCEPT 190K packets, 165M bytes)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/POSTROUTING:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/POSTROUTING:''

Chain MARK1 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/MARK1:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/MARK1:''
 1054 66052 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK set 0x1
 1054 66052 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MARK10 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/MARK10:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/MARK10:''
 3223  243K MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK set 0xa
 3223  243K ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MARK11 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/MARK11:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/MARK11:''
 1692  136K MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK set 0xb
 1692  136K ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MARK12 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/MARK12:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/MARK12:''
   60  6098 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK set 0xc
   60  6098 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MARK13 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/MARK13:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/MARK13:''
 2903 2444K MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK set 0xd
 2903 2444K ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MARK14 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/MARK14:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/MARK14:''
    2   468 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK set 0xe
    2   468 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MARK2 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/MARK2:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/MARK2:''
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK set 0x2
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MARK3 (0 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/MARK3:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/MARK3:''
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK set 0x3
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MARK4 (0 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/MARK4:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/MARK4:''
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK set 0x4
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MARK5 (0 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/MARK5:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/MARK5:''
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK set 0x5
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MARK6 (3 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/MARK6:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/MARK6:''
  163  7824 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK set 0x6
  163  7824 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MARK7 (0 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/MARK7:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/MARK7:''
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK set 0x7
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MARK8 (0 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/MARK8:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/MARK8:''
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK set 0x8
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain MARK9 (0 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `mangle/MARK9:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `mangle/MARK9:''
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK set 0x9
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

eos ~ # iptables -t nat -nvL
Chain PREROUTING (policy ACCEPT 5623 packets, 453K bytes)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/PREROUTING:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/PREROUTING:''

Chain POSTROUTING (policy ACCEPT 10 packets, 607 bytes)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/POSTROUTING:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/POSTROUTING:''
 1053 66000 SNAT_1     all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK match 0x1
    0     0 SNAT_2     all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK match 0x2
    0     0 SNAT_3     all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK match 0x3
    0     0 SNAT_4     all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK match 0x4
    0     0 SNAT_5     all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK match 0x5
  168  8064 SNAT_6     all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK match 0x6
    0     0 SNAT_7     all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK match 0x7
    0     0 SNAT_8     all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK match 0x8
    0     0 SNAT_9     all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK match 0x9
 2606  211K SNAT_10    all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK match 0xa
    0     0 SNAT_11    all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK match 0xb
    0     0 SNAT_12    all  --  *      *       0.0.0.0/0            0.0.0.0/0
MARK match 0xc

Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/OUTPUT:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/OUTPUT:''

Chain SNAT_1 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/SNAT_1:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/SNAT_1:''
 1053 66000 SNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0
to:1.1.1.139
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain SNAT_10 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/SNAT_10:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/SNAT_10:''
 2606  211K SNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0
to:2.2.2.222
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain SNAT_11 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/SNAT_11:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/SNAT_11:''
    0     0 SNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0
to:2.2.2.218
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain SNAT_12 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/SNAT_12:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/SNAT_12:''
    0     0 SNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0
to:1.1.1.139
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain SNAT_13 (0 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/SNAT_13:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/SNAT_13:''
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain SNAT_14 (0 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/SNAT_14:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/SNAT_14:''
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain SNAT_2 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/SNAT_2:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/SNAT_2:''
    0     0 SNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0
to:1.1.1.140
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain SNAT_3 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/SNAT_3:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/SNAT_3:''
    0     0 SNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0
to:1.1.1.141
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain SNAT_4 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/SNAT_4:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/SNAT_4:''
    0     0 SNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0
to:1.1.1.142
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain SNAT_5 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/SNAT_5:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/SNAT_5:''
    0     0 SNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0
to:1.1.1.137
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain SNAT_6 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/SNAT_6:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/SNAT_6:''
  168  8064 SNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0
to:2.2.2.218
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain SNAT_7 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/SNAT_7:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/SNAT_7:''
    0     0 SNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0
to:2.2.2.219
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain SNAT_8 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/SNAT_8:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/SNAT_8:''
    0     0 SNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0
to:2.2.2.220
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0

Chain SNAT_9 (1 references)
 pkts bytes target     prot opt in     out     source               
destination
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp spt:25 LOG flags 0 level 4 prefix `nat/SNAT_9:''
    0     0 LOG        tcp  --  *      *       0.0.0.0/0            0.0.0.0/0
tcp dpt:25 LOG flags 0 level 4 prefix `nat/SNAT_9:''
    0     0 SNAT       all  --  *      *       0.0.0.0/0            0.0.0.0/0
to:2.2.2.221
    0     0 ACCEPT     all  --  *      *       0.0.0.0/0            0.0.0.0/0



Logging/tcpdump from an attempt to connect to port 25 on a remote server:
Apr  9 21:55:47 eos mangle/PREROUTING:IN=eth0 OUT= 
MAC=00:40:f4:6b:6c:c1:00:01:02:1c:6f:29:08:00 SRC=192.168.1.20 DST=3.3.3.228 
LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41341 DF PROTO=TCP SPT=53218 DPT=25 
WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:55:47 eos mangle/MARK6:IN=eth0 OUT= 
MAC=00:40:f4:6b:6c:c1:00:01:02:1c:6f:29:08:00 SRC=192.168.1.20 DST=3.3.3.228 
LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41341 DF PROTO=TCP SPT=53218 DPT=25 
WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:55:47 eos nat/PREROUTING:IN=eth0 OUT= 
MAC=00:40:f4:6b:6c:c1:00:01:02:1c:6f:29:08:00 SRC=192.168.1.20 DST=3.3.3.228 
LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=41341 DF PROTO=TCP SPT=53218 DPT=25 
WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:55:47 eos mangle/FORWARD:IN=eth0 OUT=eth2 SRC=192.168.1.20 
DST=3.3.3.228 LEN=60 TOS=0x10 PREC=0x00 TTL=63 ID=41341 DF PROTO=TCP 
SPT=53218 DPT=25 WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:55:47 eos filter/FORWARD:IN=eth0 OUT=eth2 SRC=192.168.1.20 
DST=3.3.3.228 LEN=60 TOS=0x10 PREC=0x00 TTL=63 ID=41341 DF PROTO=TCP 
SPT=53218 DPT=25 WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:55:47 eos mangle/POSTROUTING:IN= OUT=eth2 SRC=192.168.1.20 
DST=3.3.3.228 LEN=60 TOS=0x10 PREC=0x00 TTL=63 ID=41341 DF PROTO=TCP 
SPT=53218 DPT=25 WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:55:47 eos nat/POSTROUTING:IN= OUT=eth2 SRC=192.168.1.20 
DST=3.3.3.228 LEN=60 TOS=0x10 PREC=0x00 TTL=63 ID=41341 DF PROTO=TCP 
SPT=53218 DPT=25 WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:55:47 eos nat/SNAT_6:IN= OUT=eth2 SRC=192.168.1.20 DST=3.3.3.228 
LEN=60 TOS=0x10 PREC=0x00 TTL=63 ID=41341 DF PROTO=TCP SPT=53218 DPT=25 
WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:55:48 eos mangle/PREROUTING:IN=eth2 OUT= 
MAC=00:08:a1:90:aa:a1:00:14:7f:03:e5:1c:08:00 SRC=3.3.3.228 DST=2.2.2.218 
LEN=60 TOS=0x00 PREC=0x00 TTL=51 ID=0 DF PROTO=TCP SPT=25 DPT=53218 
WINDOW=5792 RES=0x00 ACK SYN URGP=0
Apr  9 21:55:52 eos mangle/PREROUTING:IN=eth2 OUT= 
MAC=00:08:a1:90:aa:a1:00:14:7f:03:e5:1c:08:00 SRC=3.3.3.228 DST=2.2.2.218 
LEN=60 TOS=0x00 PREC=0x00 TTL=51 ID=0 DF PROTO=TCP SPT=25 DPT=53218 
WINDOW=5792 RES=0x00 ACK SYN URGP=0

tcpdump: listening on eth2, link-type EN10MB (Ethernet), capture size 96 bytes
21:55:47.998524 IP (tos 0x10, ttl  63, id 41341, offset 0, flags [DF], proto: 
TCP (6), length: 60) 2.2.2.218.53218 > 3.3.3.228.25: S, cksum 0x6efb 
(correct), 2404082705:2404082705(0) win 5840 <mss 1460,sackOK,timestamp 
2365113708 0,nop,wscale 2>
21:55:48.179397 IP (tos 0x0, ttl  51, id 0, offset 0, flags [DF], proto: TCP 
(6), length: 60) 3.3.3.228.25 > 2.2.2.218.53218: S, cksum 0x0b36 (correct), 
58918797:58918797(0) ack 2404082706 win 5792 <mss 1452,sackOK,timestamp 
1736175970 2365113708,nop,wscale 0>
21:55:52.175813 IP (tos 0x0, ttl  51, id 0, offset 0, flags [DF], proto: TCP 
(6), length: 60) 3.3.3.228.25 > 2.2.2.218.53218: S, cksum 0xfb9a (correct), 
58918797:58918797(0) ack 2404082706 win 5792 <mss 1452,sackOK,timestamp 
1736179965 2365113708,nop,wscale 0>
21:55:58.175073 IP (tos 0x0, ttl  51, id 0, offset 0, flags [DF], proto: TCP 
(6), length: 60) 3.3.3.228.25 > 2.2.2.218.53218: S, cksum 0xe42a (correct), 
58918797:58918797(0) ack 2404082706 win 5792 <mss 1452,sackOK,timestamp 
1736185965 2365113708,nop,wscale 0>
21:55:58.775150 IP (tos 0x0, ttl  51, id 0, offset 0, flags [DF], proto: TCP 
(6), length: 60) 3.3.3.228.25 > 2.2.2.218.53217: S, cksum 0xc92d (correct), 
4258850729:4258850729(0) ack 2314333557 win 5792 <mss 1452,sackOK,timestamp 
1736186565 2365030295,nop,wscale 0>
21:56:10.177052 IP (tos 0x0, ttl  51, id 0, offset 0, flags [DF], proto: TCP 
(6), length: 60) 3.3.3.228.25 > 2.2.2.218.53218: S, cksum 0xb54a (correct), 
58918797:58918797(0) ack 2404082706 win 5792 <mss 1452,sackOK,timestamp 
1736197965 2365113708,nop,wscale 0>


Logging/tcpdump from an attempt to connect to port 25 on the PUB2 dsl router, 
this works:
Apr  9 21:56:52 eos mangle/PREROUTING:IN=eth0 OUT= 
MAC=00:40:f4:6b:6c:c1:00:01:02:1c:6f:29:08:00 SRC=192.168.1.20 DST=2.2.2.217 
LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=34524 DF PROTO=TCP SPT=55398 DPT=25 
WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:56:52 eos mangle/MARK11:IN=eth0 OUT= 
MAC=00:40:f4:6b:6c:c1:00:01:02:1c:6f:29:08:00 SRC=192.168.1.20 DST=2.2.2.217 
LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=34524 DF PROTO=TCP SPT=55398 DPT=25 
WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:56:52 eos nat/PREROUTING:IN=eth0 OUT= 
MAC=00:40:f4:6b:6c:c1:00:01:02:1c:6f:29:08:00 SRC=192.168.1.20 DST=2.2.2.217 
LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=34524 DF PROTO=TCP SPT=55398 DPT=25 
WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:56:52 eos mangle/FORWARD:IN=eth0 OUT=eth2 SRC=192.168.1.20 
DST=2.2.2.217 LEN=60 TOS=0x10 PREC=0x00 TTL=63 ID=34524 DF PROTO=TCP 
SPT=55398 DPT=25 WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:56:52 eos filter/FORWARD:IN=eth0 OUT=eth2 SRC=192.168.1.20 
DST=2.2.2.217 LEN=60 TOS=0x10 PREC=0x00 TTL=63 ID=34524 DF PROTO=TCP 
SPT=55398 DPT=25 WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:56:52 eos mangle/POSTROUTING:IN= OUT=eth2 SRC=192.168.1.20 
DST=2.2.2.217 LEN=60 TOS=0x10 PREC=0x00 TTL=63 ID=34524 DF PROTO=TCP 
SPT=55398 DPT=25 WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:56:52 eos nat/POSTROUTING:IN= OUT=eth2 SRC=192.168.1.20 
DST=2.2.2.217 LEN=60 TOS=0x10 PREC=0x00 TTL=63 ID=34524 DF PROTO=TCP 
SPT=55398 DPT=25 WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:56:52 eos nat/SNAT_11:IN= OUT=eth2 SRC=192.168.1.20 DST=2.2.2.217 
LEN=60 TOS=0x10 PREC=0x00 TTL=63 ID=34524 DF PROTO=TCP SPT=55398 DPT=25 
WINDOW=5840 RES=0x00 SYN URGP=0
Apr  9 21:56:52 eos mangle/PREROUTING:IN=eth2 OUT= 
MAC=00:08:a1:90:aa:a1:00:14:7f:03:e5:1c:08:00 SRC=2.2.2.217 DST=2.2.2.218 
LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=46172 PROTO=TCP SPT=25 DPT=55398 WINDOW=0 
RES=0x00 ACK RST URGP=0
Apr  9 21:56:52 eos mangle/FORWARD:IN=eth2 OUT=eth0 SRC=2.2.2.217 
DST=192.168.1.20 LEN=40 TOS=0x00 PREC=0x00 TTL=63 ID=46172 PROTO=TCP SPT=25 
DPT=55398 WINDOW=0 RES=0x00 ACK RST URGP=0
Apr  9 21:56:52 eos filter/FORWARD:IN=eth2 OUT=eth0 SRC=2.2.2.217 
DST=192.168.1.20 LEN=40 TOS=0x00 PREC=0x00 TTL=63 ID=46172 PROTO=TCP SPT=25 
DPT=55398 WINDOW=0 RES=0x00 ACK RST URGP=0
Apr  9 21:56:52 eos mangle/POSTROUTING:IN= OUT=eth0 SRC=2.2.2.217 
DST=192.168.1.20 LEN=40 TOS=0x00 PREC=0x00 TTL=63 ID=46172 PROTO=TCP SPT=25 
DPT=55398 WINDOW=0 RES=0x00 ACK RST URGP=0


tcpdump: listening on eth2, link-type EN10MB (Ethernet), capture size 96 bytes
21:56:52.306357 IP (tos 0x10, ttl  63, id 34524, offset 0, flags [DF], proto: 
TCP (6), length: 60) 2.2.2.218.55398 > 2.2.2.217.25: S, cksum 0xaa49 
(correct), 2474919495:2474919495(0) win 5840 <mss 1460,sackOK,timestamp 
2365178011 0,nop,wscale 2>
21:56:52.306836 IP (tos 0x0, ttl  64, id 46172, offset 0, flags [none], proto: 
TCP (6), length: 40) 2.2.2.217.25 > 2.2.2.218.55398: R, cksum 0x7679 
(correct), 0:0(0) ack 2474919496 win 0
21:57:22.589506 IP (tos 0x0, ttl  51, id 0, offset 0, flags [DF], proto: TCP 
(6), length: 60) 3.3.3.228.25 > 2.2.2.218.53218: S, cksum 0x9a78 (correct), 
58918797:58918797(0) ack 2404082706 win 5792 <mss 1452,sackOK,timestamp 
1736270366 2365113708,nop,wscale 0>



--E.S. Johansen


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Erik S. Johansen

2006-Apr-10 09:17 UTC

head link

Re: Conntrack, nat and multipath - what is wrong here?

As a side note, if I put both dsls on eth2 and use the same ruleset, things 
start working. I still need that default route in the main table though.

--E.S. Johansen


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Erik S. Johansen

2006-Apr-10 11:16 UTC

head link

Re: Conntrack, nat and multipath - what is wrong here?

And yet another reply-to-self. Please just ignore the question and shoot me, 
i''ve spent 3 days debugging this with rpf enabled...


--E.S. Johansen


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Erik Slagter

2006-Apr-10 12:47 UTC

head link

Re: Conntrack, nat and multipath - what is wrong here?

On Mon, 2006-04-10 at 14:16 +0300, Erik S. Johansen wrote:
> And yet another reply-to-self. Please just ignore the question and shoot
me,
> i''ve spent 3 days debugging this with rpf enabled...
I''ve been bitten by the same more than once :-( Maybe that help a
little
bit ;-)


_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc

Apparently Analagous Threads

Search for more possibly parallel threads

LARTC - Apr 2006 - Conntrack, nat and multipath - what is wrong here?

Conntrack, nat and multipath - what is wrong here?

Re: Conntrack, nat and multipath - what is wrong here?

Re: Conntrack, nat and multipath - what is wrong here?

Re: Conntrack, nat and multipath - what is wrong here?

Apparently Analagous Threads