Hi,
I have typical situation, local LAN with private addresses, translated
via NAT to internet. I need to shape ingress traffic (from internet to
local LAN) in several HTB queues accorting to destination (private not
public) IP. So I need mark packets to divide them to corresponding
queue. According to http://www.docum.org/stef.coene/qos/kptd/ I thing I
have only one way how to do it, because MARK in PREROUTING is before (de)NAT
PREROUTING (de)NAT
V
FORWARD marking
V
FORWARD put to IMQ
V
HTB shapping
V
routing decision
V
output interface
It has a small problem. After PREROUTING some packets are routed to
INPUT (packets intended for this machine for local processes)
Does exists solution how to NAT and MARK in PREROUTING, but in this order?
Patrick
_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Am Monday 10 May 2004 12:10 schrieb Patrick Spousta:> So I need mark packets to divide them to corresponding queue.That''s all right so far. But the qdisc that shapes incoming traffic usually sits on your LAN device.> It has a small problem. After PREROUTING some packets are routed to > INPUT (packets intended for this machine for local processes) > > Does exists solution how to NAT and MARK in PREROUTING, but in this > order?I''m not sure if I understand what you want to do. Why do you want to mark INPUT packets? There is no qdisc/class to put them in. As for shaping incoming traffic that doesn''t get forwarded to the LAN, I haven''t found a proper solution to do that yet. So all I can do is make sure that the router doesn''t produce any traffic (e.g. don''t put a Webserver or similar services on it). Andreas _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Patrick Spousta wrote:> Hi, > I have typical situation, local LAN with private addresses, translated > via NAT to internet. I need to shape ingress traffic (from internet to > local LAN) in several HTB queues accorting to destination (private not > public) IP. So I need mark packets to divide them to corresponding > queue. According to http://www.docum.org/stef.coene/qos/kptd/ I thing I > have only one way how to do it, because MARK in PREROUTING is before > (de)NAT > > PREROUTING (de)NAT > V > FORWARD marking > V > FORWARD put to IMQ > V > HTB shapping > V > routing decision > V > output interface > > It has a small problem. After PREROUTING some packets are routed to > INPUT (packets intended for this machine for local processes) > > Does exists solution how to NAT and MARK in PREROUTING, but in this order? >If you really need to shape for local and forwarded on ingress then you use IMQ + the IMQ NAT patch and use u32 to filter on dst IP (if you are masquerading a dynamic IP mark LAN traffic and use default for local). If the traffic to local is not "bulk" ie just dns or ntp etc. then it would be less trouble to ignore it and just shape on your LAN facing interface marking on dst in postrouting or using u32 on dst - both should work, you may want to exclude traffic from server to LAN. Andy. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Andreas Klauer wrote:> Am Monday 10 May 2004 12:10 schrieb Patrick Spousta: > >>So I need mark packets to divide them to corresponding queue. > > > That''s all right so far. But the qdisc that shapes incoming traffic usually > sits on your LAN device.I think you are wrong. Shapping can sits on all interfaces, physical and logical. IMQ is logical interface.> > >>It has a small problem. After PREROUTING some packets are routed to >>INPUT (packets intended for this machine for local processes) >> >>Does exists solution how to NAT and MARK in PREROUTING, but in this >>order? > > > I''m not sure if I understand what you want to do. Why do you want to markMy linux box has 1 WAN interface (to ISP with public IP address) and 3 LAN interfaces (with private IP addresses). Only way how to shape incoming traffic is use IMG device because shapping is provided on egress. I understood that packet ''path'' looks like this eth0 -> kernel -> IMQ -> kernel -> ethX ^^^ here is ''egress'' where I can do shapping. But I need divide traffic to the corresponding queues according to real destination IP. Maybe I don''t need marking, I can only use tc filter, but it must be done in place where packet has real destination IP, ie. behind (de)NAT. To IMQ ''interface'' I put packets via iptables. Ideal in PREROUTING chain, but I think I can use only ''mange'' table and that is before ''nat'' :-( So now I''m using FORWARD chain but local traffic is going outside of shapping path> INPUT packets? There is no qdisc/class to put them in. As for shaping > incoming traffic that doesn''t get forwarded to the LAN, I haven''t found a > proper solution to do that yet. So all I can do is make sure that the > router doesn''t produce any traffic (e.g. don''t put a Webserver or similar > services on it).But it isn''t goor solution :-( Patrick> > Andreas > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ > >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Hi Andy Furniss wrote:>> Does exists solution how to NAT and MARK in PREROUTING, but in this >> order? >> > > If you really need to shape for local and forwarded on ingress then you > use IMQ + the IMQ NAT patch and use u32 to filter on dst IP (if you are > masquerading a dynamic IP mark LAN traffic and use default for local).It sounds good, but can you be more conrete?> > If the traffic to local is not "bulk" ie just dns or ntp etc. then itSometimes it is ''bulk'' - FTP etc.> would be less trouble to ignore it and just shape on your LAN facingSorry, I forgot write that I have 3 LAN interfaces, so IMQ is only way how to do it. Thanks Patrick> interface marking on dst in postrouting or using u32 on dst - both > should work, you may want to exclude traffic from server to LAN. > > Andy. >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Patrick Spousta wrote:> Hi > > Andy Furniss wrote: > >>> Does exists solution how to NAT and MARK in PREROUTING, but in this >>> order? >>> >> >> If you really need to shape for local and forwarded on ingress then >> you use IMQ + the IMQ NAT patch and use u32 to filter on dst IP (if >> you are masquerading a dynamic IP mark LAN traffic and use default for >> local). > > > It sounds good, but can you be more conrete?Using IMQ generally or a script? - mine is pretty lame, unfinished and needs netfilter patches, though I suppose it could give an indication of what to do - I am still learning HTB myself, but have got sidetracked at the moment playing with esfq. There is a new imq website www.linuximq.net from which you should be able to get imq working for whatever kernel you use. I don''t know if they include the NAT patch yet - but it''s only a couple of lines and should apply OK. If you happen to use 2.4.24 I can give urls for the patches I use. Andy. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Patrick Spousta wrote:> It looks working fine :-) I never found any details about IMQ and NAT > patch, it looks that packet processing in kernel has path > > | PREROUTING chain | > input interface -> contrack -> mangle -> nat -> imqSo which IMQ did you use - did you need to patch for NAT (there are different versions about)> ESFQ works fine, but only for ingress shapping over imq and NAT with > destination hash (== download on private IPs). I''m trying to setup ESFQ > on egress shaping for traffic from private to public IPs with source > hash (upload from private) but qdisc sits after NAT, ie. packets source > addresses are always the same public IP of external (wan) interface :-( > > I try to use imq for egress shapping (on POSTROUTING chain), may it helpsI don''t think IMQ will help - but you can mark local src in postrouting mangle OK. If you really want to use esfq, someone posted a patch on here a while back which made esfq hash on fwmark. Andy. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Andy Furniss wrote:> Patrick Spousta wrote: > >> It looks working fine :-) I never found any details about IMQ and NAT >> patch, it looks that packet processing in kernel has path >> >> | PREROUTING chain | >> input interface -> contrack -> mangle -> nat -> imq > > > So which IMQ did you use - did you need to patch for NAT (there are > different versions about)Now I''m using patches from http://www.digriz.org.uk/jdg-qos-script/ (latest version which contains patches for IMQ, IMQ+NAT, ESFQ, IPP2P, CONNMARK, also recompiledtc andlibrarie for iptables, nice package) on kernel 2.4.25. It works good.> >> ESFQ works fine, but only for ingress shapping over imq and NAT with >> destination hash (== download on private IPs). I''m trying to setup >> ESFQ on egress shaping for traffic from private to public IPs with >> source hash (upload from private) but qdisc sits after NAT, ie. >> packets source addresses are always the same public IP of external >> (wan) interface :-( >> >> I try to use imq for egress shapping (on POSTROUTING chain), may it helps > > > I don''t think IMQ will help - but you can mark local src in postroutingDo you mean manualy configured marking for many, many IP addresses? I think it isn''t right way :-( I like ESFQ for it''s source or destination hash because I don''t need to setup any filters/markers for those IPs, ESFQ creats it''s own queues for each IP. In POSTROUTING chain it normaly look like this ... -> mangle -> nat -> imq -> (output interface) I don''t understand C language so I don''t understand IMQ+NAT patch, but I''ll try to use imq for egress shapping. Maybe the patch is working identically on PRE i POST chains. Patrick> mangle OK. If you really want to use esfq, someone posted a patch on > here a while back which made esfq hash on fwmark.> > Andy. > >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Patrick Spousta wrote: <snip>>> I don''t think IMQ will help - but you can mark local src in postrouting > > > Do you mean manualy configured marking for many, many IP addresses? I > think it isn''t right way :-( I like ESFQ for it''s source or destination > hash because I don''t need to setup any filters/markers for those IPs, > ESFQ creats it''s own queues for each IP.Yes I agree - not nice for your setup, though personally the thing I don''t like about using esfq on src/dst is you loose per tcp fairness - it was less than a year ago that I was on 56K and anyone with high latency downloading from you will get their already small bandwidth squeezed out by the low latency downloaders. Maybe it''s less noticable/of an issue for your big setup anyway.> > In POSTROUTING chain it normaly look like this > > ... -> mangle -> nat -> imq -> (output interface) > > I don''t understand C language so I don''t understand IMQ+NAT patch, but > I''ll try to use imq for egress shapping. Maybe the patch is working > identically on PRE i POST chains.I only just started getting into C myself - (used motorolla 68000 assembly years ago on an atari ST - these "high level" languages are much trickier :-) ). I can see that the patch is trivial and needs an understanding of netfilter hooks more than C. I just tested with u32 to double confirm what I knew really - the patch only affects prerouting hooks. Then knowing nothing about netfilter decided to have a go at changing the egress hook - it appears to be working as expected. One caveat - some people have reported stability problems using postrouting IMQ, probably to do with dropping locally generated traffic. I and others don''t, but then I don''t leave my gateway PC up that long. Grepping my logs : Sent 3744702472 bytes 5539814 pkts (dropped 354902, overlimits 11722774) is the most I can see (most of the drops are locally generated packets ie. bittorrent running on the shaping PC). If you wan''t to give my blind and possibly stupid hack a go you just need to change near the top of drivers/net/imq.c so it looks like - static struct nf_hook_ops imq_egress_ipv4 = { { NULL, NULL}, imq_nf_hook, PF_INET, NF_IP_POST_ROUTING, NF_IP_PRI_NAT_SRC - 1 }; Rather than - static struct nf_hook_ops imq_egress_ipv4 = { { NULL, NULL}, imq_nf_hook, PF_INET, NF_IP_POST_ROUTING, NF_IP_PRI_LAST }; You could do the same for the egress ipv6 bits below it aswell. If you still have your source tree intact and use modules cd to top dir in kernel tree do make SUBDIRS=drivers/net modules which should build a new imq.o in drivers/net Backup /lib/modules/[your version]/kernel/drivers/net/imq.o and replace with new one. Take down shaping and modprobe -r imq (check it''s gone with lsmod) and restart shaper. It''s a bit of a pain that imq is unstable for some anyway - you won''t know whoose fault it is if/when it crashes :-) Andy. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/