Hi, I would like to make a QoS configuration on a linux based dsl router. It is for a server, so I want to shape outgoing traffic, incoming traffic should not be a problem as long as I have a quite assymetric connection. I would like to achieve the following goals: 1) To have one class (p2p) having all the available bandwith if there is no activity on other classes. 2) If another class (ftp server), starts to transmit, to take it all the available bandwith except for, say 20 KB/s, reserved for p2p. 3) If a third class (ssh, web), starts to transmit, again, to take all the available bandwith except for the 20KB/s for p2p and 40 KB/s for ftp. etc... I have an upload link that varies from 700kbit to 1000kbit depending on the negotiation of the line. I wouldn''t like to artificially underlimit it to work properly, remember that I''m doing the shaping on the same router, on the real (nas0) interface, so in this case the speed of the interface is the speed of the link, and I think that it''s shouldn''t be necessary to slow down the upload speed... am I wrong? The case is that I have tried to achieve this goals with several configurations of htb, but none of them has worked as I expected. I set the rate of the p2p class to 20KB/s and a higer one for ftp (40KB/s for example), and also a higher priority. But when the ftp class begin to transmit, the p2p class still remains borrowing from the paren over 20KB/s while the ftp class is below its rate. I set the rate of the parent to 1000kbit. Is this behaviour normal? Is there any other way to achive the above goals? Thanks in advance for your help and your attention. _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Javier Ors wrote:> Hi, I would like to make a QoS configuration on a linux based dsl router. It > is for a server, so I want to shape outgoing traffic, incoming traffic > should not be a problem as long as I have a quite assymetric connection. I > would like to achieve the following goals: > > 1) To have one class (p2p) having all the available bandwith if there is no > activity on other classes. > 2) If another class (ftp server), starts to transmit, to take it all the > available bandwith except for, say 20 KB/s, reserved for p2p. > 3) If a third class (ssh, web), starts to transmit, again, to take all the > available bandwith except for the 20KB/s for p2p and 40 KB/s for ftp. > etc... > > I have an upload link that varies from 700kbit to 1000kbit depending on the > negotiation of the line. I wouldn''t like to artificially underlimit it to > work properly, remember that I''m doing the shaping on the same router, on > the real (nas0) interface, so in this case the speed of the interface is the > speed of the link, and I think that it''s shouldn''t be necessary to slow down > the upload speed... am I wrong?Probably, both my PCI modems still had big buffers beyond the ppp that I shaped on. Even if nas0 did only accept traffic when it had finished transmitting I don''t think you could do what you want above.> > The case is that I have tried to achieve this goals with several > configurations of htb, but none of them has worked as I expected. I set the > rate of the p2p class to 20KB/s and a higer one for ftp (40KB/s for > example), and also a higher priority. But when the ftp class begin to > transmit, the p2p class still remains borrowing from the paren over 20KB/s > while the ftp class is below its rate. I set the rate of the parent to > 1000kbit. Is this behaviour normal? Is there any other way to achive the > above goals?The problem is that DSL sync rates are ATM rates so there are quite alot of overheads. A small eg. ack packet may be seen as 40 bytes by HTB on nas0, but will use 106 bytes (2 cells) on the wire. If you connection is highly asymmetric acks can eat a fair chunk of upload bandwidth. It affects bigger packets aswell 1500 = 32 cells = 1696 bytes ATM level. There is a solution, but you''ll need to patch your kernel and iproute2(tc), and find out what your overheads are - it depends how you connect to your ISP, see - http://ace-host.stuart.id.au/russell/files/tc/tc-atm/ To handle your varying sync rate you would need to make a script to check periodically and restart your script with the new rate. If you patch you can run almost at sync rate - back off a few kbit to allow for modem rounding down to whole cells/sec for it''s own aal0/5 QOS and if ppp lcp pings can eat a a few bits/sec if used. Also to cover yourself for restarting the script when active so any backlog can slowly drain. Andy.> > Thanks in advance for your help and your attention.
> Probably, both my PCI modems still had big buffers beyond the ppp that I > shaped on. Even if nas0 did only accept traffic when it had finished > transmitting I don''t think you could do what you want above.The problem is that DSL sync rates are ATM rates so there are quite alot> of overheads. A small eg. ack packet may be seen as 40 bytes by HTB on > nas0, but will use 106 bytes (2 cells) on the wire. If you connection is > highly asymmetric acks can eat a fair chunk of upload bandwidth. It > affects bigger packets aswell 1500 = 32 cells = 1696 bytes ATM level.Thank you for so technical answer, I suppose that this explains why the saturation speed (the one the line reaches with no shaping), is always 85%~95% of the negotiated speed that I see in the router configuration page. About the buffer size... is the same as the txqueuelength value shown in ifconfig?, it is set at 100 (packets I suppose), but it can be changed. I have no ppp encapsulation, it''s a bridged connection, so the packets go directly to this interface... There is a solution, but you''ll need to patch your kernel and> iproute2(tc), and find out what your overheads are - it depends how you > connect to your ISP, see - > > http://ace-host.stuart.id.au/russell/files/tc/tc-atm/ > > To handle your varying sync rate you would need to make a script to > check periodically and restart your script with the new rate. > > If you patch you can run almost at sync rate - back off a few kbit to > allow for modem rounding down to whole cells/sec for it''s own aal0/5 QOS > and if ppp lcp pings can eat a a few bits/sec if used. Also to cover > yourself for restarting the script when active so any backlog can slowly > drain. >Very interesting information, anyway I think that I''ll not need to do this, (I don''t even think I can, patching the kernel and the iproute2 that run on the embedded system in the router, bufff...) Hopefully, I have found a simple solution that can achive the proposed goals... It still need further testing, but I think it''ll work. I post it here, maybe someone else could be interested. This is what I''ve tested so far: prio qdisc | -------------------------- | | lower prio higher prio | | pfifo htb* | limit 60KB/s | | p2p ftp *(tbf could also have been used for this simple test) The results of this test were succesfull. I mean, with only the p2p the line was running al full saturation speed, when I started to use the ftp it reached the 60KB/s limit without problems with p2p taking the rest of the line, wich never stopped working at full speed. Given this results, I think that the following scheme will work like a charm... prio qdisc | ------------------------------------------------------ | | | low priority medium priority high priority | | | sfq htb pfifo | dynamic limit | | | | p2p ftp, web, mail... small ACK''s, ICMP, ssh... As you told me, I think that I can make a script that constantly checks the top speed on the prio (which will allways be saturated due to the p2p), and adjust the htb limit to some % of it, or substracting a fixed quantity (the quantity that will rest for the p2p when running ftp at full speed, which is what I wated to achieve in the beginning). I hope it''ll work... -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mailman.ds9a.nl/pipermail/lartc/attachments/20070829/15759ad4/attachment.htm
Javier Ors wrote:> Very interesting information, anyway I think that I''ll not need to do this, > (I don''t even think I can, patching the kernel and the iproute2 that run on > the embedded system in the router, bufff...) > Hopefully, I have found a simple solution that can achive the proposed > goals... It still need further testing, but I think it''ll work. I post it > here, maybe someone else could be interested.Ahh OK embedded. There may be hope then. I know that TI AR7 based routers don''t have a buffer beyond the device. The one I tested didn''t have any qdiscs built in apart from TIs prio/wrr, but it did work without any rate limiting. The buffer that I was talking about would not have been the 100 txqueuelen, but would have filled up before any packets got queued in that.> > This is what I''ve tested so far: > > prio qdisc > | > -------------------------- > | | > lower prio higher prio > | | > pfifo htb* > | limit 60KB/s > | | > p2p ftp > > *(tbf could also have been used for this simple test) > > The results of this test were succesfull. I mean, with only the p2p the line > was running al full saturation speed, when I started to use the ftp it > reached the 60KB/s limit without problems with p2p taking the rest of the > line, wich never stopped working at full speed. > Given this results, I think that the following scheme will work like a > charm... >Was latency OK? If you can put a simple prio on the device with ICMP (and I suppose arp to be safe since your connection is bridged) to high band and all other to low, flood it with data and you still get good latency then there isn''t much of a buffer beyond the device.> prio qdisc > | > ------------------------------------------------------ > | | | > low priority medium priority high priority > | | | > sfq htb pfifo > | dynamic limit | > | | | > p2p ftp, web, mail... small ACK''s, ICMP, ssh... > As you told me, I think that I can make a script that constantly checks the > top speed on the prio (which will allways be saturated due to the p2p), and > adjust the htb limit to some % of it, or substracting a fixed quantity (the > quantity that will rest for the p2p when running ftp at full speed, which is > what I wated to achieve in the beginning). I hope it''ll work...Yes I suppose that should be OK as long as the % wasn''t too high. Andy.
> Ahh OK embedded. There may be hope then. I know that TI AR7 based > routers don''t have a buffer beyond the device. The one I tested didn''t > have any qdiscs built in apart from TIs prio/wrr, but it did work > without any rate limiting.Then I am a lucky guy, this is mine: http://wiki.openwrt.org/OpenWrtDocs/Hardware/D-Link/DSL-G624T I didn''t looked for it, it was just the one that the ISP gave to me with the connection. I was quite surprised when I realized that it was ssh-accesible, and even more when I checked that it has htb, cbq, sfq, prio, and even red with ecn. It seems that I recieved a good machine after all. As you see they are trying to port OpenWrt to it, if they got it maybe it''ll be possible to apply russell''s patches. The buffer that I was talking about would not have been the 100> txqueuelen, but would have filled up before any packets got queued in > that.> Was latency OK? If you can put a simple prio on the device with ICMP > (and I suppose arp to be safe since your connection is bridged) to high > band and all other to low, flood it with data and you still get good > latency then there isn''t much of a buffer beyond the device.Thanks again for all the info, luckily it''s not the case, the latency is no more penalized than 20 ms in this situation, so it''s ok, no buffers. Just a question anyway...let''s suppose that we have big buffers in the device and this penalize the latency, Would this buffers also prevent from shaping with a simple prio? As I undertand, they shoudn''t as long as any packet gets dropped between the qdisc and the device buffers, they would just fill up and then the queue would propagate to the qdisc, where the packet drop and the shaping should be done. Is this not the case for all drivers/interfaces?> Yes I suppose that should be OK as long as the % wasn''t too high. >I have also observed this behaviour but I hardly undertand it. HTB works well only if you set a rate some % smaller than the congestion rate, then when you see tc -s class show, all the classes have relatively large queues (backlogged packets), and the shaping is smooth. But if you set the rate closer to the congestion rate (or higher) then you start to see empty queues in the classes and/or with few backlogged packets. I can undertand this happening in a LAN, but not in the adsl modem interface, theoretically the interface won''t dequeue packets at a higer rate that it can send them. So the queues should form anyway in the classes whatever high rate you set to the root class... Otherwise neither HTB nor the prio class would work without rate limiting (like in a LAN), so I don''t understand why in this case it works for the prio but not for HTB. Thanks again for your help and time, and please don''t feel forced to answer. I can live with this doubts, it''s not a matter of life or death... _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Javier Ors wrote:>> Ahh OK embedded. There may be hope then. I know that TI AR7 based >> routers don''t have a buffer beyond the device. The one I tested didn''t >> have any qdiscs built in apart from TIs prio/wrr, but it did work >> without any rate limiting. > > > Then I am a lucky guy, this is mine: > http://wiki.openwrt.org/OpenWrtDocs/Hardware/D-Link/DSL-G624T > I didn''t looked for it, it was just the one that the ISP gave to me with the > connection. I was quite surprised when I realized that it was ssh-accesible, > and even more when I checked that it has htb, cbq, sfq, prio, and even red > with ecn. It seems that I recieved a good machine after all. As you see they > are trying to port OpenWrt to it, if they got it maybe it''ll be possible to > apply russell''s patches.That''s handy :-) I guess the wireless ones have a bit more room for a better kernel, the non wireless one I use only has 2meg flash and 8meg ram.> > Thanks again for all the info, luckily it''s not the case, the latency is no > more penalized than 20 ms in this situation, so it''s ok, no buffers. > Just a question anyway...let''s suppose that we have big buffers in the > device and this penalize the latency, Would this buffers also prevent from > shaping with a simple prio? As I undertand, they shoudn''t as long as any > packet gets dropped between the qdisc and the device buffers, they would > just fill up and then the queue would propagate to the qdisc, where the > packet drop and the shaping should be done. Is this not the case for all > drivers/interfaces?It could work, but if the buffers are quite big two (unscaled) tcp connections won''t fill them.> > >> Yes I suppose that should be OK as long as the % wasn''t too high. >> > > I have also observed this behaviour but I hardly undertand it. HTB works > well only if you set a rate some % smaller than the congestion rate, then > when you see tc -s class show, all the classes have relatively large queues > (backlogged packets), and the shaping is smooth. But if you set the rate > closer to the congestion rate (or higher) then you start to see empty > queues in the classes and/or with few backlogged packets. > I can undertand this happening in a LAN, but not in the adsl modem > interface, theoretically the interface won''t dequeue packets at a higer rate > that it can send them. So the queues should form anyway in the classes > whatever high rate you set to the root class... Otherwise neither HTB nor > the prio class would work without rate limiting (like in a LAN), so I don''t > understand why in this case it works for the prio but not for HTB.I can''t think why that should be, I was thinking more of the P2P class being starved if the % was too high. Andy.