Hello all, I''ve been trying to test HTB performance for different link bandwidths to find potential limits and this is what I have so far: http://home.comcast.net/~msethuraman/htbtest/ Can members please go over the setup, test procedure and the results and answer a few questions? 1. Is the testing methodology okay and can the results be considered accurate? If so, is this a decent representation of behavior outside the lab? 2. Does anyone know of any limits (theoretical or observed) beyond which HTB will not work or will be inaccurate? 3. I''ve never quite understood the recommendation for setting the root HTB to 85-90% of the link. All these tests used 100%. Can someone please explain or point me to some explanations for the 90% recommendation and why it is considered necessary? If you need more information, please let me know. Thanks a lot! -- Muthu
Forwarding this to the list just so its in the archives. ---------- Forwarded message ---------- From: Jody Shumaker <jody.shumaker@gmail.com> Date: May 10, 2006 3:45 PM Subject: Re: [LARTC] HTB at 100+ Mbits/sec To: Muthukumar S <muthukumar@gmail.com>> 3. I''ve never quite understood the recommendation for setting the root > HTB to 85-90% of the link. All these tests used 100%. Can someone > please explain or point me to some explanations for the 90% > recommendation and why it is considered necessary?85-90% is just a recommendation, in reality it just needs to be slightly lower than the actual max bandwidth. The reason for this is to insure you are the bottleneck. If you aren''t the bottleneck, then it''s likely there will a queue at the the other bottleneck that will diminish the value to any shaping you do. Often the 85-90% figure is given because people may be told by their isp they have 512kbit, but in reality it rarely performs that much due to noisy lines etc. Personally I''d suggest measuring the actual max bandwidth and then set the limit below that. In my case my isp claims i have 2mbit upload, in testing I found myself to have 1840.8kbit/s upload, and thus set my max to 1836kbit/s to give myself 5kbit/s of room in case noise varies and my actual upload max varies. compare to my isp''s claimed 2mbit though, this is 91.8%. If I had just used the 90% guideline it would of been accurate. In the past though, I''ve been able to use 504kbit on a 512kbit connection because testing found it to consistently get a real throughput of around 508kbit. - Jody
Forwarding this to the list just so its in the archives. ---------- Forwarded message ---------- From: Larry Brigman <larry.brigman@gmail.com> Date: May 11, 2006 10:16 AM Subject: Re: [LARTC] HTB at 100+ Mbits/sec To: Muthukumar S <muthukumar@gmail.com> On 5/10/06, Muthukumar S <muthukumar@gmail.com> wrote:> Hello all, > > I''ve been trying to test HTB performance for different link bandwidths > to find potential limits and this is what I have so far: > > http://home.comcast.net/~msethuraman/htbtest/ > > Can members please go over the setup, test procedure and the results > and answer a few questions? > > 1. Is the testing methodology okay and can the results be considered > accurate? If so, is this a decent representation of behavior outside > the lab?Iperf has a demonstrated behavior that when running more than one copy at the same time on the same box (client side); that the timing of each will start to effect the other copies. This is a function of how Iperf does it''s timing (spin loops). If you are just wanting to test HTB, the router/bw limiter will be in the way of making accurate measurements of what HTB is doing to the traffic. Also with the router in the middle and using TCP; TCP will try to level itself to the path bw between the end points. UDP might be a better method here as you have no round trip.> > 2. Does anyone know of any limits (theoretical or observed) beyond > which HTB will not work or will be inaccurate?None that I know of. Most likely the limits will be that of the driver/hardware not allowing you to reach wire saturation (ie YMMV).> > 3. I''ve never quite understood the recommendation for setting the root > HTB to 85-90% of the link. All these tests used 100%. Can someone > please explain or point me to some explanations for the 90% > recommendation and why it is considered necessary? >The basic reasoning for limiting to < 100% of link rate is to make sure none of the choke points on the path have any reason to discard your packets. The burst and cburst parameters allow HTB to overstep the limited rate for some period of time when coming from an under-utilized link. This burst rate may be enough for your cable/DSL modem which does not have a large buffer to discard packets. Most of what I have seen here seem to indicated that a 95-98% of link rate when using rate-shaping disciplines typically works well provided you don''t have too large of burst parameters.
On 5/10/06, Muthukumar S <muthukumar@gmail.com> wrote:> First up, thanks for the response Jody. I appreciate your taking the > time to answer. > > So in essence what this means is that I will not be able to use the > maximum that the link allows if I''m shaping traffic? Please correct me > if I got this wrong - let''s say my ISP claims 512 Kbit/s upload and > real throughput varies between 450 Kbit/s and 500 Kbit/s. So if I > shaped traffic using 450 Kbit/s as the root qdisc, I would lose out on > the occasions when the line does allow more than 450 Kbit/s? >Unfortunately yes, if you want the shaping to work well you need to set it appropiately. No real way to have it vary dynamically. Basically what happens when you''re not the bottleneck is that ping times will go up as data will queue at the other bottleneck, and your bandwidth allotments will no longer accurately represent the connection. They''ll have less of an effect as TCP throttling starts having to kick in. I imagine if you designed the rules you could have the ratio between your classes still honored, and only have the increased lag or possibility for packet loss. To do this if you knew it was always atleast 450k but sometimes 500k, set the rates for all the child classes to add up to 450k, but use 500k as the highest ceiling and for the base class. Then in this case it should still continue to split the 450k evenly between the classes as you described, but still using up to the 500k when its available. Not sure how well this would work though as I''ve usually been more concerned with keeping the latency down, and set the ceil such that the majority of the time its slightly below the real bandwidth. - Jody P.S. Thanks for forwarding the email to the list, I alway forget to hit reply to all.
I''ll try using 450 K and setting a higher ceil to see how it works. Also I was wondering what limits (if any) the kernel timer resolution imposes on HTB. Thanks! Muthu On 5/11/06, Jody Shumaker <jody.shumaker@gmail.com> wrote:> On 5/10/06, Muthukumar S <muthukumar@gmail.com> wrote: > > First up, thanks for the response Jody. I appreciate your taking the > > time to answer. > > > > So in essence what this means is that I will not be able to use the > > maximum that the link allows if I''m shaping traffic? Please correct me > > if I got this wrong - let''s say my ISP claims 512 Kbit/s upload and > > real throughput varies between 450 Kbit/s and 500 Kbit/s. So if I > > shaped traffic using 450 Kbit/s as the root qdisc, I would lose out on > > the occasions when the line does allow more than 450 Kbit/s? > > > > Unfortunately yes, if you want the shaping to work well you need to > set it appropiately. No real way to have it vary dynamically. > Basically what happens when you''re not the bottleneck is that ping > times will go up as data will queue at the other bottleneck, and your > bandwidth allotments will no longer accurately represent the > connection. They''ll have less of an effect as TCP throttling starts > having to kick in. > > I imagine if you designed the rules you could have the ratio between > your classes still honored, and only have the increased lag or > possibility for packet loss. To do this if you knew it was always > atleast 450k but sometimes 500k, set the rates for all the child > classes to add up to 450k, but use 500k as the highest ceiling and > for the base class. Then in this case it should still continue to > split the 450k evenly between the classes as you described, but still > using up to the 500k when its available. Not sure how well this would > work though as I''ve usually been more concerned with keeping the > latency down, and set the ceil such that the majority of the time its > slightly below the real bandwidth. > > - Jody > > P.S. Thanks for forwarding the email to the list, I alway forget to > hit reply to all. >
> Iperf has a demonstrated behavior that when running more than one copy at the > same time on the same box (client side); that the timing of each will > start to effect > the other copies. This is a function of how Iperf does it''s timing > (spin loops).What traffic generators would you recommend? What do other members use? Has anyone used TG (http://www.postel.org/tg/)? Thanks! - Muthu
On 5/12/06, Muthukumar S <muthukumar@gmail.com> wrote:> > Iperf has a demonstrated behavior that when running more than one copy at the > > same time on the same box (client side); that the timing of each will > > start to effect > > the other copies. This is a function of how Iperf does it''s timing > > (spin loops). > > What traffic generators would you recommend? What do other members > use? Has anyone used TG (http://www.postel.org/tg/)? >I have looked at TG but would recommend D-ITG (http://www.grid.unina.it/software/ITG/) at this point as it seems to have more work recently. That said I have downloaded TG but have not ran it too much. Back to Iperf. The reason I mention the timing issue is, really only an issue when you are running multiple copies and you really want very stable inter-packet timing. If you want to use Iperf for bandwidth testing and want to run multiple copies. Get the source and find the client send routines where it is using the delay function and replace that with usleep. This change allows (in my testing) upwards of 15 copies of Iperf running at the same time without major interactions. For a single stream Iperf is the most flexible program of the bunch Just for info, here is a site that has both monitoring (mostly) tools and traffic generation tools. http://www.slac.stanford.edu/xorg/nmtf/nmtf-tools.html
On Friday 12 May 2006 12:35, Muthukumar S wrote:> > Iperf has a demonstrated behavior that when running more than one copy at > > the same time on the same box (client side); that the timing of each will > > start to effect > > the other copies. This is a function of how Iperf does it''s timing > > (spin loops). > > What traffic generators would you recommend? What do other members > use? Has anyone used TG (http://www.postel.org/tg/)?Activate chargen as tcp or udp stream at any port using inetd or xinetd. Then for tcp tests*: curl http://$host:$port/ > /dev/null or cd /dev/ && lftpget http://$host:$port/null both show traffic rate, but curl show more stats. For chargen udp test... don''t know really, maybe netcat can help. * all this is a jjo-tip[tm] -- Luciano -- Luciano
On Fri, 12 May 2006, Muthukumar S wrote:> <cut> > Also I was wondering what limits (if any) the kernel timer resolution > imposes on HTB. > <cut>The kernel timer resolution does have an impact on the precision of HTB (the delay jitter). I have done some detailed studies in my master thesis. Which is located at: http://www.adsl-optimizer.dk/thesis/. You should look at chapter 7. Read section 7.3.2 "Timer Granularity" (page 71) and section 7.3.3 "Improving Granularity" (page 73) Figure 7.8 (page 75), shows how the delay jitter bound is improved when changing the timer from HZ=100 to HZ=1500. For the discusion about, why the you need to reduce the bandwidth to less that the actual bandwidth, you should look at Chapter 6 "Achieving Queue Control". Especially if you have an ADSL link. Greeting, Jesper Brouer -- ------------------------------------------------------------------- MSc. Master of Computer Science Dept. of Computer Science, University of Copenhagen Author of http://www.adsl-optimizer.dk -------------------------------------------------------------------
On 5/14/06, Jesper Dangaard Brouer <hawk@diku.dk> wrote:> > On Fri, 12 May 2006, Muthukumar S wrote: > > > <cut> > > Also I was wondering what limits (if any) the kernel timer resolution > > imposes on HTB. > > <cut> > > The kernel timer resolution does have an impact on the precision of HTB > (the delay jitter). > > I have done some detailed studies in my master thesis. Which is located > at: http://www.adsl-optimizer.dk/thesis/.looks like the site is down, at least for me.
On Mon, 15 May 2006, Larry Brigman wrote:> On 5/14/06, Jesper Dangaard Brouer <hawk@diku.dk> wrote: >> >> On Fri, 12 May 2006, Muthukumar S wrote: >> >>> <cut> >>> Also I was wondering what limits (if any) the kernel timer resolution >>> imposes on HTB. >>> <cut> >> >> The kernel timer resolution does have an impact on the precision of HTB >> (the delay jitter). >> >> I have done some detailed studies in my master thesis. Which is located >> at: http://www.adsl-optimizer.dk/thesis/. > > looks like the site is down, at least for me.It is up again... I have just restarted apache, don''t know why apache was dead... Hilsen Jesper Brouer -- ------------------------------------------------------------------- MSc. Master of Computer Science Dept. of Computer Science, University of Copenhagen Author of http://www.adsl-optimizer.dk -------------------------------------------------------------------
Muthukumar S wrote:>> Iperf has a demonstrated behavior that when running more than one copy >> at the >> same time on the same box (client side); that the timing of each will >> start to effect >> the other copies. This is a function of how Iperf does it''s timing >> (spin loops). > > > What traffic generators would you recommend? What do other members > use? Has anyone used TG (http://www.postel.org/tg/)?Whatever you use if it''s tcp and you run multi instances I wouldn''t take much note about the numbers it gives. In your case - at high speed polling counters would be a better way to see things. I''ve done more in the way of wan speed testing and tbf/htb are not very good at simulating a slow link. HFSC is better. If you care about tcp then you need to use netem aswell to add some delay. If you don''t then because the first packets don''t get delayed the tcp sender knows you are on a lan and will/did (maybe bic) behave differently, like retransmitting a chunk of packets while the first bulk onees are sitting in your buffer waiting. Andy.