Hello, I have a Linux box with 3 network adapters: eth0 : IP:10.200.0.1/24 eth1/ppp0: IP:80.97.105.98 eth2 : IP:192.168.1.100/24 I want that all the Internet traffic goes to the eth2 interface except the smtp traffic that I want to go to the ppp0 interface. The main routing table is: 172.16.20.1 dev ppp0 proto kernel scope link src 80.97.105.98 192.168.1.0/24 dev eth2 scope link 10.200.0.0/24 dev eth0 scope link 192.168.254.0/24 dev eth1 scope link 127.0.0.0/8 dev lo scope link default via 192.168.1.1 dev eth2 I decided to use netfilter to mark the packets that leave the gateway from and to the smtp port. I do this in the OUTPUT chain of the mangle table. So, according to the docs, the marking will occur before routing for locally generated packets: $IT -t mangle -A OUTPUT -p tcp --dport smtp -j MARK --set-mark 2 $IT -t mangle -A OUTPUT -p tcp --sport smtp -j MARK --set-mark 2 Then I define a new routing table (named "smtp") and a rule to redirect smtp packets to this table. The output of "ip rule ls" is: # 0: from all lookup local # 32765: from all fwmark 2 lookup smtp # 32766: from all lookup main # 32767: from all lookup 253 In table "smtp" I defined a default route by the dev ppp0. The output of "ip route ls table smtp" is: # default dev ppp0 When I try to connect to a smtp port somewhere in the Internet, tcpdump show me that these packets go to the eth2 interface (the main table default route). I don''t know where is my mistake but it seems that the marking in the OUTPUT chain occurs AFTER and not BEFORE routing. Is this a correct behaviour? How can I solve my problem? Please help! TIA - catalin - _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hello dear Catalin, Well the only mistake you made is that you placed hte mark filters on the output of the interface. I suggest you to park them in the PREROUTING chain and not in the output. It works fine to me. Best regards Stamatis Catalin Borcea wrote:>Hello, >I have a Linux box with 3 network adapters: >eth0 : IP:10.200.0.1/24 >eth1/ppp0: IP:80.97.105.98 >eth2 : IP:192.168.1.100/24 > >I want that all the Internet traffic goes to the eth2 interface except the >smtp traffic that I want to go to the ppp0 interface. The main routing table >is: >172.16.20.1 dev ppp0 proto kernel scope link src 80.97.105.98 >192.168.1.0/24 dev eth2 scope link >10.200.0.0/24 dev eth0 scope link >192.168.254.0/24 dev eth1 scope link >127.0.0.0/8 dev lo scope link >default via 192.168.1.1 dev eth2 > > >I decided to use netfilter to mark the packets that leave the gateway from >and to the smtp port. I do this in the OUTPUT chain of the mangle table. So, >according to the docs, the marking will occur before routing for locally >generated packets: > >$IT -t mangle -A OUTPUT -p tcp --dport smtp -j MARK --set-mark 2 >$IT -t mangle -A OUTPUT -p tcp --sport smtp -j MARK --set-mark 2 > >Then I define a new routing table (named "smtp") and a rule to redirect smtp >packets to this table. The output of "ip rule ls" is: ># 0: from all lookup local ># 32765: from all fwmark 2 lookup smtp ># 32766: from all lookup main ># 32767: from all lookup 253 > >In table "smtp" I defined a default route by the dev ppp0. The output of "ip >route ls table smtp" is: ># default dev ppp0 > >When I try to connect to a smtp port somewhere in the Internet, tcpdump show >me that these packets go to the eth2 interface (the main table default >route). I don''t know where is my mistake but it seems that the marking in >the OUTPUT chain occurs AFTER and not BEFORE routing. Is this a correct >behaviour? How can I solve my problem? Please help! > >TIA >- catalin - > > >_______________________________________________ >LARTC mailing list / LARTC@mailman.ds9a.nl >http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ > > >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hello, I tried to mark the packets in the PREROUTING chain but still doesn''t work. Now the packets are no marked anymore when they go out by the eth2 interface. When I marked them in the OUTPUT chain they arrived also to the eth2 interface but marked. According to the docs the PREROUTING chain is not traversed by locally generated packets so, I don''t know how this works for you. Maybe you have forwarded packets and not locally generated packets. - catalin - ----- Original Message ----- From: "???????? ?????" <skekes@pylones.gr> To: "Catalin Borcea" <catalin@electricant.ambra.ro> Cc: <lartc@mailman.ds9a.nl> Sent: Thursday, July 17, 2003 9:04 AM Subject: Re: [LARTC] OUTPUT chain marking after or before routing?> Hello dear Catalin, > Well the only mistake you made is that you placed hte mark filters on > the output of the interface. > I suggest you to park them in the PREROUTING chain and not in the > output. It works fine to me. > Best regards > Stamatis > Catalin Borcea wrote: > > >Hello, > >I have a Linux box with 3 network adapters: > >eth0 : IP:10.200.0.1/24 > >eth1/ppp0: IP:80.97.105.98 > >eth2 : IP:192.168.1.100/24 > > > >I want that all the Internet traffic goes to the eth2 interface exceptthe> >smtp traffic that I want to go to the ppp0 interface. The main routingtable> >is: > >172.16.20.1 dev ppp0 proto kernel scope link src 80.97.105.98 > >192.168.1.0/24 dev eth2 scope link > >10.200.0.0/24 dev eth0 scope link > >192.168.254.0/24 dev eth1 scope link > >127.0.0.0/8 dev lo scope link > >default via 192.168.1.1 dev eth2 > > > > > >I decided to use netfilter to mark the packets that leave the gatewayfrom> >and to the smtp port. I do this in the OUTPUT chain of the mangle table.So,> >according to the docs, the marking will occur before routing for locally > >generated packets: > > > >$IT -t mangle -A OUTPUT -p tcp --dport smtp -j MARK --set-mark 2 > >$IT -t mangle -A OUTPUT -p tcp --sport smtp -j MARK --set-mark 2 > > > >Then I define a new routing table (named "smtp") and a rule to redirectsmtp> >packets to this table. The output of "ip rule ls" is: > ># 0: from all lookup local > ># 32765: from all fwmark 2 lookup smtp > ># 32766: from all lookup main > ># 32767: from all lookup 253 > > > >In table "smtp" I defined a default route by the dev ppp0. The output of"ip> >route ls table smtp" is: > ># default dev ppp0 > > > >When I try to connect to a smtp port somewhere in the Internet, tcpdumpshow> >me that these packets go to the eth2 interface (the main table default > >route). I don''t know where is my mistake but it seems that the marking in > >the OUTPUT chain occurs AFTER and not BEFORE routing. Is this a correct > >behaviour? How can I solve my problem? Please help! > > > >TIA > >- catalin - > > > > > >_______________________________________________ > >LARTC mailing list / LARTC@mailman.ds9a.nl > >http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ > > > > > > > > > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Well catalin, try to add theese rules with the in the prerouting chain but bind theese rules with the interfaces you have. Without binding netfilter rules with interfaces it will not work, and you''ll get the results you allready got. Catalin Borcea wrote:>Hello, >I tried to mark the packets in the PREROUTING chain but still doesn''t work. >Now the packets are no marked anymore when they go out by the eth2 >interface. When I marked them in the OUTPUT chain they arrived also to the >eth2 interface but marked. According to the docs the PREROUTING chain is not >traversed by locally generated packets so, I don''t know how this works for >you. Maybe you have forwarded packets and not locally generated packets. > >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
But how can I bind these rules to a interface when I don''t know to what interface the locally generated packets will arrive? In fact, this is the purpose of marking the packets: to route them to the ppp0 interface. - catalin - ----- Original Message ----- From: "???????? ?????" <skekes@pylones.gr> To: "Catalin Borcea" <catalin@electricant.ambra.ro> Cc: <lartc@mailman.ds9a.nl> Sent: Thursday, July 17, 2003 9:37 AM Subject: Re: [LARTC] OUTPUT chain marking after or before routing?> Well catalin, try to add theese rules with the in the prerouting chain > but bind theese rules with the interfaces you have. > Without binding netfilter rules with interfaces it will not work, and > you''ll get the results you allready got. > Catalin Borcea wrote: > > >Hello, > >I tried to mark the packets in the PREROUTING chain but still doesn''twork.> >Now the packets are no marked anymore when they go out by the eth2 > >interface. When I marked them in the OUTPUT chain they arrived also tothe> >eth2 interface but marked. According to the docs the PREROUTING chain isnot> >traversed by locally generated packets so, I don''t know how this worksfor> >you. Maybe you have forwarded packets and not locally generated packets. > > > > > > > >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Then you have to bind theese rules to both of your ethernet interfaces assuming that the smtp traffic you want to mark arrives on both of your ethernet interfaces. To do that you have to specify netfilter rules once for your first interface and once for your second interface. I know that it looks complicated a little bit but it''ll work. Catalin Borcea wrote:>But how can I bind these rules to a interface when I don''t know to what >interface the locally generated packets will arrive? In fact, this is the >purpose of marking the packets: to route them to the ppp0 interface. > > >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
have u tried putting it on the FORWARD chain?? K But how can I bind these rules to a interface when I don''t know to what interface the locally generated packets will arrive? In fact, this is the purpose of marking the packets: to route them to the ppp0 interface. - catalin - ----- Original Message ----- From: "???????? ?????" <skekes@pylones.gr> To: "Catalin Borcea" <catalin@electricant.ambra.ro> Cc: <lartc@mailman.ds9a.nl> Sent: Thursday, July 17, 2003 9:37 AM Subject: Re: [LARTC] OUTPUT chain marking after or before routing? > Well catalin, try to add theese rules with the in the prerouting chain > but bind theese rules with the interfaces you have. > Without binding netfilter rules with interfaces it will not work, and > you''ll get the results you allready got. > Catalin Borcea wrote: > > >Hello, > >I tried to mark the packets in the PREROUTING chain but still doesn''t work. > >Now the packets are no marked anymore when they go out by the eth2 > >interface. When I marked them in the OUTPUT chain they arrived also to the > >eth2 interface but marked. According to the docs the PREROUTING chain is not > >traversed by locally generated packets so, I don''t know how this works for > >you. Maybe you have forwarded packets and not locally generated packets. > > > > > > > > _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ _________________________________________________________________ The new MSN 8: advanced junk mail protection and 2 months FREE* http://join.msn.com/?page=features/junkmail _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Catalin,>When I try to connect to a smtp port somewhere in the Internet, tcpdump show >me that these packets go to the eth2 interface (the main table default >route). I don''t know where is my mistake but it seems that the marking in >the OUTPUT chain occurs AFTER and not BEFORE routing. Is this a correct >behaviour? How can I solve my problem? Please help!According to my reading of the KPTD (and my understanding), packets generated on the local machine have already been routed by the time the OUTPUT chain is traversed. See: http://www.docum.org/stef.coene/qos/kptd/ : have u tried putting it on the FORWARD chain?? Unfortunately the FORWARD chain will not work if these are locally generated packets. I see two potential approaches to this problem: - invert your logic; main routing table uses ppp0 gateway IP as default gateway, mark all traffic passing through your router box, and use "ip rule add fwmark $MARK table $INTERNET" with another routing table for the Internet-bound traffic. - send all locally generated traffic via ppp0; "ip rule add iif lo table smtp" and watch all traffic generated on the local machine leave via ppp0. You''ll want to add the locally connected networks to table smtp. -Martin -- Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
lartc@manchotnetworks.net
2003-Jul-17 16:50 UTC
Re: OUTPUT chain marking after or before routing?
Hi Martin, Catalin, Chijioke, This subject intrigues me greatly and is closely related to a post of just a few days ago: <snip from my original post>> >+----------------------+ +---------------+ > >| eth1 192.168.1.1 |------------| 192.168.1.250 | > >| eth1:1 192.168.1.101 | | | > >+----------------------+ +---------------+ > > > > > >iptables --append OUTPUT --table mangle --jump MARK --set-mark 0x2 > >ip rule add fwmark 0x2 table 2 > >ip route add 192.168.1.0/24 dev eth1 src 192.168.1.101 table 2 > >ip route flush cache > > > > > >telnet 192.168.1.250 ; and tcpdump gives src ip address as > >192.168.1.1 > > > > > >ip rule add to 192.168.1.250 table 2 > >ip route flush cache > > > > > >telnet 192.168.1.250 ; and tcpdump gives src ip address as > >192.168.1.101> According to my reading of the KPTD (and my understanding), packets > generated on the local machine have already been routed by the time the > OUTPUT chain is traversed. See: > > http://www.docum.org/stef.coene/qos/kptd/i have spent alot of time looking at this diagram and don''t understand what happens when. curiously, to my post patrick McHardy was kind enough to test and: On Sun, 2003-07-13 at 23:43, Patrick McHardy wrote:> I tested your setup and it works fine (with 2.5 though). Are you sure > you have > CONFIG_IP_ROUTE_FWMARK enabled for your running kernel ? ip rule won''t > give errors if not ..very interesting, and i have yet to make it work here, although i haven''t debugged it yet> : have u tried putting it on the FORWARD chain?? > > Unfortunately the FORWARD chain will not work if these are locally > generated packets.yup.> > I see two potential approaches to this problem: > > - invert your logic; main routing table uses ppp0 gateway IP as default > gateway, mark all traffic passing through your router box, and use > "ip rule add fwmark $MARK table $INTERNET" with another routing > table for the Internet-bound traffic.martin, this is pure genius> > - send all locally generated traffic via ppp0; "ip rule add iif lo > table smtp" and watch all traffic generated on the local machine leave > via ppp0. You''ll want to add the locally connected networks to table > smtp.can you comment why this is -- ip rule to xxx.xxx.xxx.xxx table n works, and iptables fwmark y table n doesn''t? is it because OUTPUT checked the rule while the packet was "generated" locally, but not after it was marked? 1000 thanks charles _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
----- Original Message ----- From: "Martin A. Brown" <mabrown-lartc@securepipe.com> To: "Chijioke Kalu" <kchijioke@msn.com> Cc: <catalin@electricant.ambra.ro>; <lartc@mailman.ds9a.nl> Sent: Thursday, July 17, 2003 6:55 PM Subject: Re: [LARTC] OUTPUT chain marking after or before routing?> Catalin, > > >When I try to connect to a smtp port somewhere in the Internet, tcpdumpshow> >me that these packets go to the eth2 interface (the main table default > >route). I don''t know where is my mistake but it seems that the marking in > >the OUTPUT chain occurs AFTER and not BEFORE routing. Is this a correct > >behaviour? How can I solve my problem? Please help! > > According to my reading of the KPTD (and my understanding), packets > generated on the local machine have already been routed by the time the > OUTPUT chain is traversed. See: > > http://www.docum.org/stef.coene/qos/kptd/ >I''m very confused now. Look what is written in the iptables man page: ############# mangle This table is used for specialized packet alteration. It has two built-in chains: PREROUTING (for altering incoming packets before routing) and OUTPUT (for altering locally-generated packets before routing). ###################### So how it is? OUTPUT marks packets AFTER or BEFORE routing?> I see two potential approaches to this problem: > > - invert your logic; main routing table uses ppp0 gateway IP as default > gateway, mark all traffic passing through your router box, and use > "ip rule add fwmark $MARK table $INTERNET" with another routing > table for the Internet-bound traffic.This approach is harder for me because this is a working gateway and I don''t wan''t to disturb the users with my tests. But, it is a very good idea and maybe I will try it.> > - send all locally generated traffic via ppp0; "ip rule add iif lo > table smtp" and watch all traffic generated on the local machine leave > via ppp0. You''ll want to add the locally connected networks to table > smtp.I also tried that and it works. But I don''t want to send all locally generated traffic to ppp0. In fact I want only the smtp traffic on ppp0. The Web traffic (including Squid generated, which is locally generated) must go to eth2. Thank you for your reply, - catalin - _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
: I''m very confused now. Look what is written in the iptables man page: : : ############# : mangle This table is used for specialized packet alteration. It has two : built-in : chains: PREROUTING (for altering incoming packets before : routing) and OUTPUT : (for altering locally-generated packets before routing). : ###################### : : So how it is? OUTPUT marks packets AFTER or BEFORE routing? Hmph. It certainly seems that somebody is lying! I defer to those who know the kernel better on such matters, but my understanding is in line with the KPTD. : > I see two potential approaches to this problem: : > : > - invert your logic; main routing table uses ppp0 gateway IP as default : > gateway, mark all traffic passing through your router box, and use : > "ip rule add fwmark $MARK table $INTERNET" with another routing : > table for the Internet-bound traffic. : : This approach is harder for me because this is a working gateway and I : don''t wan''t to disturb the users with my tests. But, it is a very good : idea and maybe I will try it. I can certainly respect that--one note, though this still doesn''t solve the problem of the squid traffic which is also locally generated. : > - send all locally generated traffic via ppp0; "ip rule add iif lo : > table smtp" and watch all traffic generated on the local machine leave : > via ppp0. You''ll want to add the locally connected networks to table : > smtp. : I also tried that and it works. But I don''t want to send all locally : generated traffic to ppp0. In fact I want only the smtp traffic on : ppp0. The Web traffic (including Squid generated, which is locally : generated) must go to eth2. Can you force traffic from squid to leave with a particular source IP address? (I do not know how you would do this with any SMTP client, but you could equally do the same with your SMTP client software.) Here''s the idea: - force squid traffic to be initiated with a source IP of 192.168.1.100 - "ip rule add iif lo table smtp" - "ip rule add iif lo from 192.168.1.100 table main" This still means that other locally generated traffic will leave via your ppp0. So, instead........ - force smtp traffic to be initiated with a source IP of 80.97.105.98 - "ip rule add iif lo from 192.168.1.100 table smtp" All other traffic will go via the default route in table main, which will push the traffic out your eth2 link. Good luck, -Martin -- Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Charles, : can you comment why this is -- : : ip rule to xxx.xxx.xxx.xxx table n : : works, and : : iptables fwmark y table n : : doesn''t? is it because OUTPUT checked the rule while the packet was : "generated" locally, but not after it was marked? I can certainly make such a comment. The RPDB is consulted for every *new* route lookup. Any source/dest,(tos/fwmark/iif) tuple which is not in the route cache will be looked up. The lookup process checks the RPDB and any routing tables specified by the RPDB. This all happens before the OUTPUT chain for locally generated packets. So, locally generated packets marked in the OUTPUT chain have already been routed. As I mentioned before, I will defer to those who know the kernel code better, but my understanding is exactly in line with the KPTD [1]. I have also written in more detail on the route selection process [2]. Best, -Martin [1] http://www.docum.org/stef.coene/qos/kptd/ [2] http://linux-ip.net/html/routing-selection.html -- Martin A. Brown --- SecurePipe, Inc. --- mabrown@securepipe.com _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
lartc@manchotnetworks.net
2003-Jul-19 07:45 UTC
Re: OUTPUT chain marking after or before routing?
Hello Martin, Thanks very much for your explanation.> So, locally generated packets marked in the OUTPUT chain have already been > routed.This is really quite sad isn''t it? I was attempting the following for locally generated packets: iptables --append OUTPUT --table mangle --match owner \ --uid-owner 500 --jump MARK --set-mark 0x5 ip rule fwmark 0x5 table 5; etc ... By matching the process uid or gid, I was counting on being able to policy route based on who was asking -- quite a neat solution actually. ip rule doesn''t allow to match a uid/gid, and from your explanation, it would be hard to imagine. I don''t suppose you have a way around this?? Cheers Charles _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Leonardo Balliache
2003-Jul-20 15:19 UTC
Re: OUTPUT chain marking after or before routing?
Hi, At 08:04 a.m. 18/07/03 +0300, you wrote:>----- Original Message ----- >From: "Martin A. Brown" <mabrown-lartc@securepipe.com> >To: "Chijioke Kalu" <kchijioke@msn.com> >Cc: <catalin@electricant.ambra.ro>; <lartc@mailman.ds9a.nl> >Sent: Thursday, July 17, 2003 6:55 PM >Subject: Re: [LARTC] OUTPUT chain marking after or before routing? > > > > Catalin, > > > > >When I try to connect to a smtp port somewhere in the Internet, tcpdump >show > > >me that these packets go to the eth2 interface (the main table default > > >route). I don''t know where is my mistake but it seems that the marking in > > >the OUTPUT chain occurs AFTER and not BEFORE routing. Is this a correct > > >behaviour? How can I solve my problem? Please help! > > > > According to my reading of the KPTD (and my understanding), packets > > generated on the local machine have already been routed by the time the > > OUTPUT chain is traversed. See: > > > > http://www.docum.org/stef.coene/qos/kptd/ > > > >I''m very confused now. Look what is written in the iptables man page: > >############# > mangle This table is used for specialized packet alteration. It has two >built-in > chains: PREROUTING (for altering incoming packets before >routing) and OUTPUT > (for altering locally-generated packets before routing). >###################### > >So how it is? OUTPUT marks packets AFTER or BEFORE routing?Just before "output routing". OUTPUT is for locally generated packets. These packets are also to be routed (output routing). OUTPUT mangle marks "locally generated" packets just before they are "output routing". Perhaps confussion is because also input routing exists where a decision is taken: is this packet for this host or it has just to be forwarded? Read Stef´s remarks on the diagram: Output routing : the local process selects a source address and a route. This route is attached to the packet and used later. Best regards, Leonardo Balliache Practical QoS http://opalsoft.net/qos _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/