Dear Community, sorry for the somewhat dumb question. Maybe someone has any pointer to how to setup the RED queue to mark pakets with ECN. In particular what are appropriate parameter settings for limit, min, max, etc. All my trials end up with either "RTNETLINK answers: Invalid argument", although the command line (at least for me) looks fine in regard to what is said on the man page. "tc qdisc replace dev lo red limit 10kb min 2kb max 8kb avpkt 1000 burst 12 ecn" being one such example. I am using kernel version 2.6.16.29 and the most recent iproute2 package. Thanks in advance, Daniel
Hi Daniel,>Dear Community, > >sorry for the somewhat dumb question. Maybe someone has any pointer >to how to setup the RED queue to mark pakets with ECN. In particular >what are appropriate parameter settings for limit, min, max, etc. > >All my trials end up with either "RTNETLINK answers: Invalid >argument", although the command line (at least for me) looks fine in >regard to what is said on the man page. > >"tc qdisc replace dev lo red limit 10kb min 2kb max 8kb avpkt 1000 >burst 12 ecn" being one such example.You did not specify where to attach the qdisc: (see the keyword "root") tc qdisc replace dev lo root \ red limit 10kb min 2kb max 8kb avpkt 1000 burst 12 ecn>I am using kernel version 2.6.16.29 and the most recent iproute2 >package. > >Thanks in advance, >DanielRegards /Christian [ http://benve.info ]
Daniel Schaffrath wrote:> Dear Community, > > sorry for the somewhat dumb question. Maybe someone has any pointer to > how to setup the RED queue to mark pakets with ECN. In particular what > are appropriate parameter settings for limit, min, max, etc.I have never really played with red - but I wouldn''t start by putting it on lo - I don''t think you will get a backlog. Even on root of an eth there will be quite big buffer to fill before it gets any backlog. You are best using it as a child of something that rate limits tbf/hfsc/htb. There is a rate parameter which I guess you should use - but it''s not to actually do the rate limiting. As for ECN, it''s off by default on Linux, so I don''t think it will be much use unless you have control over the network. I just turned it on on two of my PCs - echo 1 > /proc/sys/net/ipv4/tcp_ecn and did a test with red and it does work and save some drops. Andy.
Andy Furniss schrieb:> Daniel Schaffrath wrote: >> Dear Community, >> >> sorry for the somewhat dumb question. Maybe someone has any >> pointer to how to setup the RED queue to mark pakets with ECN. In >> particular what are appropriate parameter settings for limit, >> min, max, etc. > > I have never really played with red - but I wouldn''t start by > putting it on lo - I don''t think you will get a backlog. Even on > root of an eth there will be quite big buffer to fill before it > gets any backlog.Could you explain this a bit more in detail, why does it not work on root of an device? I tried it with various configurations and indeed it does not work. Even if the incoming interface is much faster then the outgoing interface I can''t get the red queue to drop or mark packets. Packets are always dropped somewhere else?> > You are best using it as a child of something that rate limits > tbf/hfsc/htb. There is a rate parameter which I guess you should > use - but it''s not to actually do the rate limiting.Thanks for the hint! It seems to work that way, I used this: tc qdisc add dev wifi0 root handle 1 tbf rate 36mbit burst 5kb latency 100ms peakrate 54mbit minburst 1540 tc qdisc add dev wifi0 parent 1: red limit 10000 min 2000 max 5000 avpkt 1000 burst 2 probability .2 ecn Then suddenly i get marked packets: qdisc red 8003: dev wifi0 parent 1: limit 10000b min 2000b max 5000b ecn Sent 5913561 bytes 3938 pkt (dropped 0, overlimits 6 requeues 2611) rate 0bit 0pps backlog 0b 0p requeues 2611 marked 6 early 0 pdrop 0 other 0 But question remains: Why does it not work with root qdisc? And 6 packets are still a bit few? Ingoing interface is 100 mbit, outgoing link on wifi0 is about 5 mbit...> As for ECN, it''s off by default on Linux, so I don''t think it will > be much use unless you have control over the network. I just turned > it on on two of my PCs - > > echo 1 > /proc/sys/net/ipv4/tcp_ecn > > and did a test with red and it does work and save some drops.Could you show your configuration or is it too complex? How many packets got marked? Thank you. Best regards, Arnd
Arnd Hannemann wrote:> > Could you explain this a bit more in detail, why does it not work on > root of an device? > I tried it with various configurations and indeed it does not work. > Even if the incoming interface is much faster then the outgoing > interface I can''t get the red queue to drop or mark packets. Packets > are always dropped somewhere else?I suppose exactly what happens depends on the device drivers and the type of device. On my 100mbit eth I probably could get RED to work a bit, but I would have to fill a buffer of about 300 MTU size packets first, so it would never be right as such. I see you have wireless, I don''t have any wireless, but I guess the drivers may drop/shape without ever backlogging the root device - or maybe you don''t generate enough traffic to fill the buffer. I think shaping on wireless is going to be hard. Single duplex so you can''t really know the bandwidth, random loss due to errors etc. You can probably do better than doing nothing, though - at the expense of sacrificing bandwidth. It may be possible to use ifb to have ingress and egress share the same bandwidth.> Thanks for the hint! It seems to work that way, I used this: > > tc qdisc add dev wifi0 root handle 1 tbf rate 36mbit burst 5kb latency > 100ms peakrate 54mbit minburst 1540 > tc qdisc add dev wifi0 parent 1: red limit 10000 min 2000 max 5000 > avpkt 1000 burst 2 probability .2 ecn > > Then suddenly i get marked packets: > qdisc red 8003: dev wifi0 parent 1: limit 10000b min 2000b max 5000b ecn > Sent 5913561 bytes 3938 pkt (dropped 0, overlimits 6 requeues 2611) > rate 0bit 0pps backlog 0b 0p requeues 2611 > marked 6 early 0 pdrop 0 other 0 > > But question remains: Why does it not work with root qdisc? > And 6 packets are still a bit few? > Ingoing interface is 100 mbit, outgoing link on wifi0 is about 5 mbit...You could try try 5 or 4mbit as tbf rate rather than 36 :-)> Could you show your configuration or is it too complex? How many > packets got marked?When I first did it I just pasted Daniels config into a script I had that used ifb/hfsc, limited it to 500kbit and tried with and without the ecn parameter. The buffer sizes are too small and my hfsc limiter would fail if not on ifb so i won''t post that one :-) I tried again using the bandwidth parameter (which I called rate earlier) and with bigger buffers at 5mbit using 5 netperfs I can get nearly no drops. (if you run multiple netperfs from a script make sure and addresses are in /etc/hosts and 0.0.0.0 is aswell as it seems to insist on doing dns lookups - which can delay some of them) I guess wifi like eth uses arp - so using tbf on root may drop some arp, but just to test I put it on root of my 100mbit eth. tc qdisc add dev eth0 root handle 1: tbf rate 5mbit burst 2k limit 100k tc qdisc add dev eth0 parent 1: red limit 100kb min 10kb max 60kb avpkt 1000 burst 12 probability .2 bandwidth 5mbit ecn After 100 seconds of traffic - qdisc tbf 1: rate 5000Kbit burst 2Kb lat 82.7ms Sent 62495756 bytes 41328 pkt (dropped 1, overlimits 87189 requeues 0) rate 0bit 0pps backlog 0b 0p requeues 0 qdisc red 8002: parent 1: limit 100Kb min 10Kb max 60Kb ecn Sent 62495756 bytes 41328 pkt (dropped 1, overlimits 2412 requeues 87189) rate 0bit 0pps backlog 0b 0p requeues 87189 marked 2411 early 1 pdrop 0 other 0 I wonder if the drop was arp - I also tried again while pinging and got 7% loss. Pings without ecn gave 13% loss. The same test but without the ecn parameter - [root@amd /home/andy/Qos]# tc -s qdisc ls dev eth0 qdisc tbf 1: rate 5000Kbit burst 2Kb lat 82.7ms Sent 62473422 bytes 41315 pkt (dropped 2320, overlimits 90861 requeues 0) rate 0bit 0pps backlog 0b 0p requeues 0 qdisc red 8001: parent 1: limit 100Kb min 10Kb max 60Kb Sent 62473422 bytes 41315 pkt (dropped 2320, overlimits 2320 requeues 90861) rate 0bit 0pps backlog 0b 0p requeues 90861 marked 0 early 2320 pdrop 0 other 0 Andy.
Hi Andy, Andy Furniss schrieb:> Arnd Hannemann wrote: > >> >> Could you explain this a bit more in detail, why does it not work on >> root of an device? >> I tried it with various configurations and indeed it does not work. >> Even if the incoming interface is much faster then the outgoing >> interface I can''t get the red queue to drop or mark packets. Packets >> are always dropped somewhere else? > > I suppose exactly what happens depends on the device drivers and the > type of device. On my 100mbit eth I probably could get RED to work a > bit, but I would have to fill a buffer of about 300 MTU size packets > first, so it would never be right as such. > I see you have wireless, I don''t have any wireless, but I guess the > drivers may drop/shape without ever backlogging the root device - or > maybe you don''t generate enough traffic to fill the buffer.ah, I see. I think now I understand more how it works. The kernel hands down the skb to the device driver, and for RED to work on the root device, the device driver has to explicitly handle the skb, back, or at least communicate that it is busy.> > > I tried again using the bandwidth parameter (which I called rate > earlier) and with bigger buffers at 5mbit using 5 netperfs I can get > nearly no drops. (if you run multiple netperfs from a script make sure > and addresses are in /etc/hosts and 0.0.0.0 is aswell as it seems to > insist on doing dns lookups - which can delay some of them) > > I guess wifi like eth uses arp - so using tbf on root may drop some > arp, but just to test I put it on root of my 100mbit eth. > > tc qdisc add dev eth0 root handle 1: tbf rate 5mbit burst 2k limit 100k > > tc qdisc add dev eth0 parent 1: red limit 100kb min 10kb max 60kb > avpkt 1000 burst 12 probability .2 bandwidth 5mbit ecn > > After 100 seconds of traffic - > > qdisc tbf 1: rate 5000Kbit burst 2Kb lat 82.7ms > Sent 62495756 bytes 41328 pkt (dropped 1, overlimits 87189 requeues 0) > rate 0bit 0pps backlog 0b 0p requeues 0 > > qdisc red 8002: parent 1: limit 100Kb min 10Kb max 60Kb ecn > Sent 62495756 bytes 41328 pkt (dropped 1, overlimits 2412 requeues > 87189) > rate 0bit 0pps backlog 0b 0p requeues 87189 > marked 2411 early 1 pdrop 0 other 0 > > I wonder if the drop was arp - I also tried again while pinging and > got 7% loss. Pings without ecn gave 13% loss. > > > The same test but without the ecn parameter - > > [root@amd /home/andy/Qos]# tc -s qdisc ls dev eth0 > > qdisc tbf 1: rate 5000Kbit burst 2Kb lat 82.7ms > Sent 62473422 bytes 41315 pkt (dropped 2320, overlimits 90861 > requeues 0) > rate 0bit 0pps backlog 0b 0p requeues 0 > > qdisc red 8001: parent 1: limit 100Kb min 10Kb max 60Kb > Sent 62473422 bytes 41315 pkt (dropped 2320, overlimits 2320 requeues > 90861) > rate 0bit 0pps backlog 0b 0p requeues 90861 > marked 0 early 2320 pdrop 0 other 0 >Andy, thanks a lot for your efforts. Its good to see that in principal ECN really works and can save DROPs ;-) Best regards, Arnd