LARTC - Jan 2004 - Precedence of iptables chain, local routing table and newly created routing table

Hi, 

I been trying on ip rule fwmark and iptables MARK. 

I will show my testing in detail, but my ultimate question is why ONLY marking
in Mangle OUTPUT tables works, but not others?

Network Diagram

------------ 192.168.250.197 eth0 LINUX ROUTER eth1 192.168.8.88
------------------ 192.168.8.112 eth0 Windows XP Client

Steps (performed on LINUX ROUTER)
(1) Delete route to 192.168.8.0 from local routing table on
(2) Add route to 192.168.8.0 at table test2
(3) Mark packet with --set-mark 3 at MANGLE OUTPUT table
(4) Forward all packet marked 3 to table test2 using ip rule fwmark
(5) Do a ip ro flush cache
(6) Ping from 192.168.8.112 to 192.168.8.88 is successful


[root@son-ag webauth]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.250.0   0.0.0.0         255.255.255.0   U     0      0        0 eth0
127.0.0.0       0.0.0.0         255.0.0.0       U     0      0        0 lo
0.0.0.0         192.168.250.254 0.0.0.0         UG    0      0        0 eth0


[root@son-ag webauth]# ip route show table test2
192.168.8.0/24 via 192.168.8.88 dev br0


[root@son-ag webauth]# iptables -t mangle -L
Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination
MARK       all  --  anywhere             anywhere           MARK set 0x3


[root@son-ag webauth]# ip ru
0:      from all lookup local
32764:  from all fwmark        3 lookup test2
32766:  from all lookup main
32767:  from all lookup 253


I wish to know why is that ONLY marking at OUTPUT table works?
The network setup is for testing purpose, I wish to know the precedence of
iptables chains, local routing table and newly created table (e.g. test2)

Looking at the iptables chain diagram, my guess is MARKING at mangle INPUT or
mangle PREROUTING should work as well.
When packet comes off from wire, I mark it with 3 at mangle PREROUTING. Since it
is a ping to 192.168.8.88, it should be a local process.
Then the ping is successful. But from my testing, no. 

Another possiblity is packet is route to test2 routing table after mangle OUTPUT
and before mandle POSTROUTING. I am getting confuse :)

Please advice. Thank you

Kaiwen