Hi all! I try to make port based routing, because a have two connections to the internet. My router is a "one disk floppy router for linux". It is a big router project www.fli4l.de. I try also to make a opt, it is like a plugin for this router. This project uses Kernel 2.2.19 compiled with libc5 (because it is small and you can use one floppy disk). At the moment, iproute2 is not implementated. So i downloaded a old libc5 distribution and the kernel 2.2.19. I compiled the kernel with the iproute 2 related options. I also compiled on this system the iproute2 package. On my router (at the moment on harddisk) i added the new kernel and the "ip" from iproute2. Everything seems to work. I tried a destination host based routing - works percect. But not the port based routing. How is my configuration? Three nics: eth0 - 192.168.0.0/24 - local LAN - masqeraded eth1 - ppp0 - dialup! - A-DSL Provider eth2 - static IP - S-DSL Provider - routet to another router. This do not work for me: Treid to route all SSH Traffic to eth2 and WEB Traffic to ppp0: first, i mark the pakets with ipchains in the input chain [mark 1 is eth2 | mark 2 is ppp0]: ipchains -A input -p tcp -s 192.168.0.0/24 -d 0/0 22 -m 1 ipchains -A input -p tcp -s 192.168.0.0/24 -d 0/0 80 -m 2 second, i added two rules: echo 200 t-dsl >> /etc/iproute2/rt_tables echo 201 s-dsl >> /etc/iproute2/rt_tables ip ru add fwmark 1 table s-dsl ip ru add fwmark 2 table a-dsl at last, i setup the routes: ip ro add 0/0 dev eth2 table s-dsl ip ro add 0/0 dev ppp0 table a-dsl The maqerading is also setup: ipchains -A forward -s 192.168.0.0/24 -j MASQ There nothing happens. If i try to connect with ssh or http from a other host, nothing happens. If a setup this with two "desternation host rules" it works. I think there is something wrong with ipchains and the marking of packets or ip rule can not read the mark. How can i test, if the packtes get marked? Anybody knows a other solution? Is there a mistake? Best Regards Marco _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Marco, : ip ro add 0/0 dev eth2 table s-dsl : ip ro add 0/0 dev ppp0 table a-dsl You need to specify a default gateway here, or else you are telling your box to route 0/0 directly out the interface--which means it will arp for every address on the Internet on your local ethernet! ip route add 0/0 via x.x.x.x table s-dsl ip route add 0/0 via x.x.x.x table a-dsl should do it. You can use the "dev $DEVNAME" if you wish. : The maqerading is also setup: : ipchains -A forward -s 192.168.0.0/24 -j MASQ : How can i test, if the packtes get marked? Look at the verbose ipchains output ("ipchains -nvL forward") output to see if the usage counter on the particular chain increments. -Martin -- Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hi Martin! I send this mail with a other E-Mal account - because I am now at home, but I am Marco!! Okay, I tried this. But is does not work. It is very strange, because I made a tcpdump and the result shows it is the masq? The configuration: ipchains -A input -p icmp -s 192.168.0.0/24 -m 2 ip ru add fwmark 2 table 10 ip route add default via x.x.x.x dev eth2 table 10 ipchains -A forward -s 192.168.0.0/24 -j MASQ * x.x.x.x is the default gateway! here the tcpdump on eth2 during a ping from internal 192.168.0.31 to a host in the internet (ping 62.154.89.102 - 4 times timeout): tcpdump: listening on eth2 19:20:28.532089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request 19:20:28.572089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply 19:20:33.532089 arp who-has x.x.x.x tell y.y.y.y 19:20:33.532089 arp reply x.x.x.x is-at 0:0:c0:b1:a9:90 19:20:33.852089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request 19:20:33.882089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply 19:20:38.852089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request 19:20:38.892089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply 19:20:43.862089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request 19:20:43.902089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply y.y.y.y = the ip of eth2 x.x.x.x = the gateway You can see, the ping goes out and returns on the eth2 interface. But it will not be masqueraded to the internal host 192.168.0.31. On this host, I started the ping. Other strange thing: after the return of the first reply, there is a pause of 5 seconds. After that comes a arp request. And anything else: if I delete the rule fwmark 2 table 10, the client (192.168.0.31) shows during a ping to outside: 192.168.0.1 (ip of eth0): no route to host The ip rule seems to work and the ip route too because the icmp packet goes out and comes back. But why will it not be route to the internal host, which has sent it? I really do not know what is wrong here. If I do: ip ru add default via x.x.x.x dev eth2 Everything works well - everything goes over eth2. You wrote:> : ip ro add 0/0 dev eth2 table s-dsl > : ip ro add 0/0 dev ppp0 table a-dsl > >You need to specify a default gateway here, or else you are tellingyour>box to route 0/0 directly out the interface--which means it will arpfor>every address on the Internet on your local ethernet! > >ip route add 0/0 via x.x.x.x table s-dsl >ip route add 0/0 via x.x.x.x table a-dsl > >should do it. You can use the "dev $DEVNAME" if you wish. > > : The maqerading is also setup: > : ipchains -A forward -s 192.168.0.0/24 -j MASQ > > : How can i test, if the packtes get marked? > >Look at the verbose ipchains output ("ipchains -nvL forward") output to >see if the usage counter on the particular chain increments.And with ipchains -nvL, i can see the packets will be marked in the input chain. This seems to work too. Hope anybody have I idea. Best Regards Marco _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hi Marco, Let''s work through this together--your description is good--I''m a bit puzzled too. : ipchains -A input -p icmp -s 192.168.0.0/24 -m 2 : ip ru add fwmark 2 table 10 : ip route add default via x.x.x.x dev eth2 table 10 : ipchains -A forward -s 192.168.0.0/24 -j MASQ You don''t write a rule to allow explicitly for the returning echo-reply packets. If you have a default policy of DENY on your input chain, then the packet can appear on eth2 (see your tcpdump below) but still be denied. If you want to make sure that the policy never gets called (easier to see if packets are hitting the last rule than if they are hitting the chain policy): ipchains -A input -i eth2 -j DENY -l Now you''ll have a chain that logs the packets and has a counter--this will make it easier to see if ipchains is part of the problem. : here the tcpdump on eth2 during a ping from internal 192.168.0.31 to a : host in the internet (ping 62.154.89.102 - 4 times timeout): : : tcpdump: listening on eth2 : 19:20:28.532089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request : 19:20:28.572089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply : 19:20:33.532089 arp who-has x.x.x.x tell y.y.y.y : 19:20:33.532089 arp reply x.x.x.x is-at 0:0:c0:b1:a9:90 : 19:20:33.852089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request : 19:20:33.882089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply : 19:20:38.852089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request : 19:20:38.892089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply : 19:20:43.862089 y.y.y.y > L-EB1.L.DE.net.dtag.de: icmp: echo request : 19:20:43.902089 L-EB1.L.DE.net.dtag.de > y.y.y.y: icmp: echo reply : : y.y.y.y = the ip of eth2 : x.x.x.x = the gateway : You can see, the ping goes out and returns on the eth2 interface. But it : will not be masqueraded to the internal host 192.168.0.31. Minor note: I''d say "de-masqueraded" here because the packet was masqueraded on the outbound trip, and will now be associated with the existing masqueraded connection. To view the masquerading table (if you have any doubt about whether the entry exists or not): ipchains -nML : On this host, I started the ping. : Other strange thing: after the return of the first reply, there is a : pause of 5 seconds. After that comes a arp request. Now that strikes me as very odd. I don''t have an easy answer for this one...unless you started the ping like this: ping -i 5 L-EB1.L.DE.net.dtag.de : And anything else: if I delete the rule fwmark 2 table 10, the client : (192.168.0.31) shows during a ping to outside: : 192.168.0.1 (ip of eth0): no route to host What does "ip rule show" tell you after you have removed the above rule? : The ip rule seems to work and the ip route too because the icmp packet : goes out and comes back. But why will it not be route to the internal : host, which has sent it? See my speculation above....ipchains interference? : I really do not know what is wrong here. : If I do: : ip ru add default via x.x.x.x dev eth2 : Everything works well - everything goes over eth2. By the way, you are using "ip route flush cache" every time you make changes to the routing tables/RPDB, right? OK, so: - remove all extraneous ip rule entries (from the RPDB) - add the one RPDB rule you want - check all your routing tables - ip route flush cache - try again from the internal host - check the counters on the ipchains Also, could you show me what "ip rule show" says on the affected box? -Martin -- Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Aigh! I think I may have spotted the problem. Your routing table number 10 doesn''t know anything about 192.168.0.0/24 does it? Make sure that each routing table has routes for the destinations it is supposed to be able to reach! : ipchains -A input -p icmp -s 192.168.0.0/24 -m 2 : ip ru add fwmark 2 table 10 : ip route add default via x.x.x.x dev eth2 table 10 : ipchains -A forward -s 192.168.0.0/24 -j MASQ : * x.x.x.x is the default gateway! -Martin -- Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hi Martin! Thanks for your feedback. Okay, i will try to go step by step.> : ipchains -A input -p icmp -s 192.168.0.0/24 -m 2 > : ip ru add fwmark 2 table 10 > : ip route add default via x.x.x.x dev eth2 table 10 > : ipchains -A forward -s 192.168.0.0/24 -j MASQ > >You don''t write a rule to allow explicitly for the returning echo-reply >packets. If you have a default policy of DENY on your input chain,then>the packet can appear on eth2 (see your tcpdump below) but still be >denied.I made every new try a ipchains -F - there was no other chain(s).>If you want to make sure that the policy never gets called (easier tosee>if packets are hitting the last rule than if they are hitting the chain >policy): > > ipchains -A input -i eth2 -j DENY -l > >Now you''ll have a chain that logs the packets and has a counter--thiswill>make it easier to see if ipchains is part of the problem.Okay, it seems there is a problem. In this DENY chain I get after every ping 4 more packets (one ping - 4 tries). It seems ipchains deny the incoming icmp packets on eth2. But why? I tried also to specify the source ip with some other chains, and it is the packet, that comes from the host 62.154.89.102 - exactly the packet I am waiting for. A ipchains -nML shows a open masq connection to the host, I ping''d: IP masquerading entries prot expire source destination ports ICMP 00:57.85 192.168.0.31 62.154.89.102 512 (61009) -> 8>Now that strikes me as very odd. I don''t have an easy answer for this >one...unless you started the ping like this: > > ping -i 5 L-EB1.L.DE.net.dtag.deI used the ip address.> : And anything else: if I delete the rule fwmark 2 table 10, theclient> : (192.168.0.31) shows during a ping to outside: > : 192.168.0.1 (ip of eth0): no route to host > >What does "ip rule show" tell you after you have removed the aboverule? 0: from all lookup local 32766: from all lookup main 32767: from all lookup 253 Yes, it is okay - if I delete the rule, the network could not be reached, because I have no default route ( also in the table main). It was only a _simple_ test to see if ip rules works. With this rule: 0: from all lookup local 32765: from all fwmark 2 lookup 10 32766: from all lookup main 32767: from all lookup 253 there is a timeout. It shows me, the marking of packets works and the ip rules can see the marked packets.>By the way, you are using "ip route flush cache" every time you make >changes to the routing tables/RPDB, right?Yes, i do. ---- cut ---- your next Mail ---- cut ---->Aigh! I think I may have spotted the problem. > >Your routing table number 10 doesn''t know anything about 192.168.0.0/24 >does it? > >Make sure that each routing table has routes for the destinations it is >supposed to be able to reach! > > : ipchains -A input -p icmp -s 192.168.0.0/24 -m 2 > : ip ru add fwmark 2 table 10 > : ip route add default via x.x.x.x dev eth2 table 10 > : ipchains -A forward -s 192.168.0.0/24 -j MASQ > : * x.x.x.x is the default gateway!Well, if I look into the rules table I see: 0: from all lookup local 32765: from all fwmark 2 lookup 10 32766: from all lookup main 32767: from all lookup 253 Okay! If I start the ping, the icmp packet comes into the input chain with source address ex. 192.168.0.31 (my local host). (ipchains -A input -p icmp -s 192.168.0.0/24 -m 2) ipchains looks: - is the source address in network 192.168.0.0/24? Yes. - is a icmp packet? Yes. Okay it will be marked with 2. Now the ip rules number 32765 (all fwmark 2 lookup 10) see it and activates routing table 10. The routing table 10: default via x.x.x.x dev eth2 x.x.x.x is the gateway. Okay, it goes out on eth2 to the gateway - to the target host. It answers a icmp packet. It arrives on eth2. Not it is going hot :) Normaly it should do the following: Comes into the input chain. Ipchains looks: - is the source address in network 192.168.0.0/24? No. The source address it _not_ in the net 192.168.0.0/24, so it will not be marked. Now it is ip rules turn: 0: from all lookup local 32765: from all fwmark 2 lookup 10 32766: from all lookup main 32767: from all lookup 253 okay, not local, not marked with 10 - is comes into the main routing table! And this is the main routing table: 217.5.98.39 dev ppp0 proto kernel scope link src 80.131.187.122 n.n.n.n/28 dev eth2 proto kernel scope link src y.y.y.y 192.168.1.0/24 dev eth1 proto kernel scope link src 192.168.1.1 192.168.0.0/24 dev eth0 proto kernel scope link src 192.168.0.1 n.n.n.n/28 is the net and y.y.y.y the ip address of eth2 In the main routing table, there is the net 192.168.0.0/24, so it should go on! But okay. This is not the problem. It seems, ipchains DENY this packet. But why? Here a ipchains -L: Chain input (policy ACCEPT): target prot opt source destination ports - icmp ------ 192.168.0.0/24 anywhere any -> any DENY all ----l- anywhere anywhere n/a Chain forward (policy DENY): target prot opt source destination ports MASQ all ------ 192.168.0.0/24 anywhere n/a Chain output (policy ACCEPT): The deny chain, is your chain to monitor :) Without it (the deny chain) it is exactly the same siduation. Wth denys ipchains this incoming packet on eth2? Marco _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hi again, Marco, : I made every new try a ipchains -F - there was no other chain(s). Got it. : Okay, it seems there is a problem. In this DENY chain I get after every : ping 4 more packets (one ping - 4 tries). : It seems ipchains deny the incoming icmp packets on eth2. But why? : I tried also to specify the source ip with some other chains, and it is : the packet, that comes from the host 62.154.89.102 - exactly the packet : I am waiting for. : : A ipchains -nML shows a open masq connection to the host, I ping''d: : : IP masquerading entries : prot expire source destination ports : ICMP 00:57.85 192.168.0.31 62.154.89.102 512 (61009) -> 8 All is well. : 0: from all lookup local : 32765: from all fwmark 2 lookup 10 : 32766: from all lookup main : 32767: from all lookup 253 : : there is a timeout. It shows me, the marking of packets works and the ip : rules can see the marked packets. Looks right. : >By the way, you are using "ip route flush cache" every time you make : >changes to the routing tables/RPDB, right? : : Yes, i do. This is just a common problem--so I wanted to ask. : >Aigh! I think I may have spotted the problem. : >Your routing table number 10 doesn''t know anything about 192.168.0.0/24 : >does it? : >Make sure that each routing table has routes for the destinations it is : >supposed to be able to reach! : > : ipchains -A input -p icmp -s 192.168.0.0/24 -m 2 : > : ip ru add fwmark 2 table 10 : > : ip route add default via x.x.x.x dev eth2 table 10 : > : ipchains -A forward -s 192.168.0.0/24 -j MASQ : > : * x.x.x.x is the default gateway! : Well, if I look into the rules table I see: : 0: from all lookup local : 32765: from all fwmark 2 lookup 10 : 32766: from all lookup main : 32767: from all lookup 253 <I snipped much of your mail with which I agree> : But okay. This is not the problem. : It seems, ipchains DENY this packet. But why? : : Here a ipchains -L: : Chain input (policy ACCEPT): : target prot opt source destination : ports : - icmp ------ 192.168.0.0/24 anywhere any : -> any : DENY all ----l- anywhere anywhere n/a : Chain forward (policy DENY): : target prot opt source destination : ports : MASQ all ------ 192.168.0.0/24 anywhere n/a : Chain output (policy ACCEPT): I was suggesting the "ipchains -A input -j DENY -l" chain to make sure that any packet passing through is explicitly logged and dropped instead of implicitly. I''m sure you''ll see lots of DENY traffic in your /var/log/messages when using this rule, and things definitely won''t work. Sorry if that was at all unclear--this was intended as a diagnosing tool. : The deny chain, is your chain to monitor :) : Without it (the deny chain) it is exactly the same siduation. : Wth denys ipchains this incoming packet on eth2? It doesn''t look to me like the input chain is your problem, but rather your forward chain. The default policy is deny. Try changing that to allow specifically what you want to allow. You could always try that very same diagnosing ipchains rule in your forward chain, i.e. "ipchains -A forward -j DENY -l". Then you''ll see that the de-masqueraded packet is denied passing through the forward chain. (At least that''s my guess....) This, of course, is the beauty of using iptables--much less worrying with iptables rules than with ipchains rules (in general), but you are using kernel 2.2.19, I believe, so iptables is not an option for you. Let us know how you fare, -Martin -- Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Next :)>You could always try that very same diagnosing ipchains rule in your >forward chain, i.e. "ipchains -A forward -j DENY -l". Then you''ll see >that the de-masqueraded packet is denied passing through the forward >chain. (At least that''s my guess....)I did. I understand the deny chain now - it was my mistake. In the forward chain, I added the deny chain: ipchains -A input -i eth2 -j DENY -l But no packets arrive there. I write it down, the short version: Chain input (policy ACCEPT): target prot opt source destination ports - icmp ------ 192.168.0.0/24 anywhere any -> any Chain forward (policy ACCEPT): target prot opt source destination ports MASQ all ------ 192.168.0.0/24 anywhere n/a DENY all ----l- anywhere anywhere n/a Chain output (policy ACCEPT): So the default policy is accept. With a ping of 4 tries, the forward - MASQ chain added 4 pakets and the icmp mark chain added also 4 packets. But no one in the DENY chain. The same with the deny chain in the input chain: ipchains -A forward -j DENY -l Chain input (policy ACCEPT): target prot opt source destination ports - icmp ------ 192.168.0.0/24 anywhere any -> any DENY all ----l- anywhere anywhere n/a Chain forward (policy ACCEPT): target prot opt source destination ports MASQ all ------ 192.168.0.0/24 anywhere n/a Chain output (policy ACCEPT): There with the same ping, 4 packets added in the MASQ, in the icmp _and_ in the input deny chain. Hmm, if I don''t make anything wrong, the packets get lost after the input and before the forward chain. What do you think? Now it is time to go to bed, its 11:30pm here. I am at home tomorrow at 5pm CET (hope so) and will try again - so long to it works, the next day is free for me, so I have the whole night tomorrow. Marco _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/