Hi All, I have few queries in reference of HTB manuals on link http://luxik.cdi.cz/~devik/qos/htb/htbman.htm. 1> Is "default" is must while attaching queue discipline HTB to device? My observation for above question is, when I do following 4 command then I m able to reach to all hosts on eth1. But if I remove "default 12" from second command then I cannot reach to hosts on eth1, but I can reach to the host on other device (eth0, eth2). This occurs after I create extra class to serve as the root (i.e. creating class having parent 1:1, 4th command) tc qdisc del dev eth1 root tc qdisc add dev eth1 root handle 1: htb default 12 tc class add dev eth1 parent 1: classid 1:1 htb rate 100kbps ceil 100kbps burst 2k tc class add dev eth1 parent 1:1 classid 1:11 htb rate 10kbps ceil 100kbps burst 2k tc class add dev eth1 parent 1:1 classid 1:12 htb rate 60kbps ceil 100kbps burst 2k Ya, this reachability can be restored only after delition of root class, i.e. classid 1:1. I''m taking this as a feature but is it a bug? 2> Manual states that "... ... ... Now we can optionally attach queuing disciplines to the leaf classes... ... ... tc qdisc add dev eth0 parent 1:10 handle 20: pfifo limit 5... ... ...". If I not going to do that still I''m fine with my rule set, as written "optional". Then is there benefit or payoff against this leaf-class? 3> A Small but nice document about Cross device sharing on link http://luxik.cdi.cz/~devik/qos/htb/htbmeas1.htm. Cant this limitation of slot be increased? 4> It would be nice if someone help me by providing statistics over number of queues and filter. What is maximum limit of these two for a PIII 700 with 256 MB RAM? How much maximum bandwidth can be shaped? Thanks in Advance. -- Sumit
> 2> Manual states that "... ... ... Now we can optionally attach queuing > disciplines to the leaf classes... ... ... tc qdisc add dev eth0 parent > 1:10 handle 20: pfifo limit 5... ... ...". If I not going to do that still > I''m fine with my rule set, as written "optional". Then is there benefit or > payoff against this leaf-class?Each class has a qdisc attached. By default, this is a FIFO one (First In, First Out). You can replace it with a SFQ qdisc. A SFQ qdisc will treath each flow with the same priority. So one flow can not eat all bandwidth.> 3> A Small but nice document about Cross device sharing on link > http://luxik.cdi.cz/~devik/qos/htb/htbmeas1.htm. Cant this limitation of > slot be increased?Can''t find that page.> > 4> It would be nice if someone help me by providing statistics over number > of queues and filter. What is maximum limit of these two for a PIII 700 > with 256 MB RAM? How much maximum bandwidth can be shaped?I think you can shape a lot of traffic with this setup, but I don''t know exactly. Stef -- stef.coene@docum.org More QOS info : http://www.docum.org/ Title : "Using Linux as bandwidth manager"
> > 1> Is "default" is must while attaching queue discipline HTB to device?no it is optional.> My observation for above question is, when I do following 4 command then I > m able to reach to all hosts on eth1. But if I remove "default 12" from > second command then I cannot reach to hosts on eth1, but I can reach to the > host on other device (eth0, eth2). This occurs after I create extra class to > serve as the root (i.e. creating class having parent 1:1, 4th command) > tc qdisc del dev eth1 root > tc qdisc add dev eth1 root handle 1: htb default 12 > tc class add dev eth1 parent 1: classid 1:1 htb rate 100kbps ceil 100kbps > burst 2k > tc class add dev eth1 parent 1:1 classid 1:11 htb rate 10kbps ceil 100kbps > burst 2k > tc class add dev eth1 parent 1:1 classid 1:12 htb rate 60kbps ceil 100kbps > burst 2k > > Ya, this reachability can be restored only after delition of root class, > i.e. classid 1:1. I''m taking this as a feature but is it a bug?depends on filters you are using. Without "default" all unclassified traffic is lost (like ARP packets) unless you have some catch-all filter which cares about such traffic. The bahaviour changed in htb2 patch - not even without default all unclassified traffic goes into virtual X:0 class which is pass-thru internal fifo IIRC. It is funny that I can''t remember how I implemented latest semantic here. ... well I just checked the source and I''m right with the fifo. You can look at stats from tc -s class ... to see fate of your packets.> 2> Manual states that "... ... ... Now we can optionally attach queuing > disciplines to the leaf classes... ... ... tc qdisc add dev eth0 parent 1:10 > handle 20: pfifo limit 5... ... ...". If I not going to do that still I''m > fine with my rule set, as written "optional". Then is there benefit or > payoff against this leaf-class?this general tc issue was replied by Stef to you :) (thanx Stef)> 3> A Small but nice document about Cross device sharing on link > http://luxik.cdi.cz/~devik/qos/htb/htbmeas1.htm. Cant this limitation of > slot be increased?the slots was removed from latest code and link above is obsolete (where you have found it ?) ;) The slots concept was too much of hack and there was too many races on SMP (unfixable ones). The IMQ device is superior for Cross device sharing and it is a LOT simpler ;) See http://luxik.cdi.cz/~devik/qos/> 4> It would be nice if someone help me by providing statistics over number > of queues and filter. What is maximum limit of these two for a PIII 700 with > 256 MB RAM? How much maximum bandwidth can be shaped?I''ve tried on my pII 375, I was able to shape 100Mbit with 5 classes (3 leaves). But both CBQ and HTB has O(N) worst case complexity where N is no of classes. I just finished new HTB design (with the same interface of course) and it should be faster in this way. The I will do some serious testing on ratw vs. no of classes. Just to note that speed depends ONLY on number of ACTIVE classes (active flows). Those without data enqueued are idle and don''t decrease throughtput. Also limiting is packet rate not byte rate .. The limit is somewhere at 30000 packets per second for both CBQ and HTB with 5 classes which is cca 500Mbit for 1.5k packets (on my PII). devik
-----Original Message----- From: Martin Devera [mailto:devik@cdi.cz]>> 3> A Small but nice document about Cross device sharing on link >> http://luxik.cdi.cz/~devik/qos/htb/htbmeas1.htm. Can?t this limitation of >> slot be increased?> the slots was removed from latest code and link above is obsolete (where > you have found it ?) ;)I can still find this link despite of refreshing page in my browser, May some cache server running in between.>The IMQ device is superior for Cross device sharing and it is a LOT simpler;)> See http://luxik.cdi.cz/~devik/qos/COOL IMQ. How much throughput will be reduced or latency will be added while using IMQ? As in every flow it adds/increase one hope internally?>> 4> It would be nice if someone help me by providing statistics overnumber>> of queues and filter. What is maximum limit of these two for a PIII 700with>> 256 MB RAM? How much maximum bandwidth can be shaped?> I''ve tried on my pII 375, I was able to shape 100Mbit with 5 classes (3 > leaves). But both CBQ and HTB has O(N) worst case complexity where N is > no of classes. I just finished new HTB design (with the same interface > of course) and it should be faster in this way. > The I will do some serious testing on ratw vs. no of classes.> Just to note that speed depends ONLY on number of ACTIVE classes (active > flows). Those without data enqueued are idle and don''t decrease > throughtput.Presently I''m running setup with 200 queues (200 RED leaves and 200 u32 classifiers), Thus basically a seperate queue for every individual in my organization. Ya I never knew about the dependency of throughput on active flows. I''m planing to go for 400 queues. Sometime before Documentations on Devik''s home page stat that RED is not going to work with HTB and will simply eat CPU cycles. I couldn''t locate it now anywhere? Does it works with new release of HTB?> Also limiting is packet rate not byte rate .. The limit is somewhere > at 30000 packets per second for both CBQ and HTB with 5 classes which > is cca 500Mbit for 1.5k packets (on my PII).But ultimately its going to limit on byte transfer, isn''t it? -- Sumit
> > the slots was removed from latest code and link above is obsolete (where > > you have found it ?) ;) > I can still find this link despite of refreshing page in my browser, May > some cache server running in between.No cache .. the page is still on my server but it is treated as obsolete. I just marked the link to it as such ;)> COOL IMQ. How much throughput will be reduced or latency will be added > while using IMQ? As in every flow it adds/increase one hope internally?Yes it is one enqueue and dequeue for qdisc. So that is will add latency as dictated by the qdisc .. So that there should be only negligible unwanted performance hit.> > Just to note that speed depends ONLY on number of ACTIVE classes (active > > flows). Those without data enqueued are idle and don''t decrease > > throughtput. > Presently I''m running setup with 200 queues (200 RED leaves and 200 u32 > classifiers), Thus basically a seperate queue for every individual in my > organization. Ya I never knew about the dependency of throughput on active > flows. I''m planing to go for 400 queues. Sometime before Documentations on > Devik''s home page stat that RED is not going to work with HTB and will > simply eat CPU cycles. I couldn''t locate it now anywhere? Does it works with > new release of HTB?I can''t remember that I would have such note on my pages. But basically HTB,CBQ and TBF are exclusive options. They don''t fit well one into another. It is because they are all absolute rate dependent and all are non-work conserving. RED is also dependant on absolute rate but it is work-conserving. So that it can be used as CBQ''s or HTB''s leaf and there should be ho hard problem with it. However I don''t know RED sources well so that I can''t predict some obscure problems.> > Also limiting is packet rate not byte rate .. The limit is somewhere > But ultimately its going to limit on byte transfer, isn''t it?Yes as end effect it is. devik
> Subject: RE: [LARTC] exploring HTBI expected someone else to say this before, but since nobody has yet ... > > > Just to note that speed depends ONLY on number of ACTIVE classes (active > > > flows). Those without data enqueued are idle and don''t decrease > > > throughtput. First, the classification time could be pretty significant with hundreds of filters. You could minimize this by building a balanced tree of classes, so that it would only take log(#classes) filters, but that would be a lot of trouble to build and maintain. > > Presently I''m running setup with 200 queues (200 RED leaves and 200 u32 > > classifiers), Thus basically a seperate queue for every individual in my > > organization. Ya I never knew about the dependency of throughput on active > > flows. I''m planing to go for 400 queues. Sometime before Documentations on > > Devik''s home page stat that RED is not going to work with HTB and will > > simply eat CPU cycles. I couldn''t locate it now anywhere? Does it works with > > new release of HTB? I don''t think this is the problem HTB was meant to solve. I think what you really want is the (often suggested/requested) modified version of sfq that hashes only on source or destination address (depending on whether you use it for the queue going in to or out of your local network). If your addresses are consecutive then the "hash" function could be particularly good (fast and no collisions). This would not allow you to offer different amounts of service to different addresses, but that could also be added easily (though it would then reintroduce the inconvenience of maintaining this data in the configuration file).
> > > > Just to note that speed depends ONLY on number of ACTIVE classes (active > > > > flows). Those without data enqueued are idle and don''t decrease > > > > throughtput. > First, the classification time could be pretty significant with > hundreds of filters. You could minimize this by building a balanced > tree of classes, so that it would only take log(#classes) filters, but > that would be a lot of trouble to build and maintain.I''m not sure. How balanced tree of classes could help you with big number of filters ? These are unrelated IMHO. When filter determines class number it is looked using hash table. If you use u32 with 200 nodes in tree and htb then classification time will be about 5 steps for each packet...> > > Presently I''m running setup with 200 queues (200 RED leaves and 200 u32 > > > classifiers), Thus basically a seperate queue for every individual in my > > > organization. Ya I never knew about the dependency of throughput on active > > I don''t think this is the problem HTB was meant to solve. > I think what you really want is the (often suggested/requested) > modified version of sfq that hashes only on source or destination > address (depending on whether you use it for the queue going in to orRight. I think so too :) HTB can be useful in this way if you want different parameters (rate,prio) per host .. But we definitely need the changed sfq. The change is trivial but I''m too deep in new htb just now :( devik
Martin Devera writes: > I''m not sure. How balanced tree of classes could help you with big number > of filters ? These are unrelated IMHO. When filter determines class number > it is looked using hash table. > If you use u32 with 200 nodes in tree and htb then classification time > will be about 5 steps for each packet... What do you do if you have 100 rules of form: if source ip = ip1 then class1 if source ip = ip2 then class2 if source ip = ip3 then class3 ... I assume you have to try them until one matches or you run out. However, I believe that you now support filters at interior classes. This means that at the top you could do something like if source ip in 10.0.0.0 - 10.0.0.50 then class1 if source ip in 10.0.0.51 - 10.0.0.99 then class2 and then in class1 put filters: if source ip in 10.0.0.0 - 10.0.0.25 then class3 if source ip in 10.0.0.26 - 10.0.0.50 then class3 etc. Then you''d only have to match o(log(n)) filters. > > I don''t think this is the problem HTB was meant to solve. > > I think what you really want is the (often suggested/requested) > > modified version of sfq that hashes only on source or destination > > address (depending on whether you use it for the queue going in to or > > Right. I think so too :) HTB can be useful in this way if you want > different parameters (rate,prio) per host .. > But we definitely need the changed sfq. The change is trivial but > I''m too deep in new htb just now :( I can''t promise any time soon either. If someone else wants to work on this I''d like to send some (reverse engineered) documentation to include in future sfq - might also help you to make further changes.
> What do you do if you have 100 rules of form: > if source ip = ip1 then class1 > if source ip = ip2 then class2 > if source ip = ip3 then class3 > ... > I assume you have to try them until one matches or you run out.If these are different filters then yes. But you can use u32''s hash to match them all in one step.> However, I believe that you now support filters at interior classes. > This means that at the top you could do something like > if source ip in 10.0.0.0 - 10.0.0.50 then class1 > if source ip in 10.0.0.51 - 10.0.0.99 then class2 > and then in class1 put filters: > if source ip in 10.0.0.0 - 10.0.0.25 then class3 > if source ip in 10.0.0.26 - 10.0.0.50 then class3 > etc. Then you''d only have to match o(log(n)) filters.Oh yes. But the you can''t use classes for balancing rates - because you will need balanced binary tree .. But if you use inner classes then you are right it will be always better (faster). But for large ip count I''d use u32 hash table. devik