Hi all, I am having some fun with traffic shaping, and have run into an interesting situation. Here is South Africa, most internet links are heavily oversubscribed, which means that in most cases the local link is _not_ the bottleneck, and shaping on the local link does not help that much... Does anybody have some tips on shaping such links? How can you get interractive traffic if you don''t know how much bandwidth to reserve for it? How can you give fair access to a link if you don''t know what the link capacity is? Are there perhaps some tools to monitor retransmissions to try and determine congestion levels, and from that adjust shaped bandwidth? Am I perhaps missing something simple in this scenario? Thanks! -justin _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Quick answer is: you can''t. You need to know the bandwidth so that you can control the queue. You bring up some interesting theoretical ideas about monitoring the overall congestion level and forcing back-offs when you sense that you''re reaching the current level, though. That would sort of involve dynamically adjusting the shaping rules. As far as I know there is nothing in existance that does this, but its an interesting idea to think about. If you were going to try something like this I think you''d need some sort of reliable indicator of what the current congestion is like... perhaps some steady ping to use as your "canary in the coal mine". Then set up several shaping scripts that assume different levels of bandwidth, and depending on the current "canary ping", either upgrade or downgrade your assumed bandwidth by calling the appropriate script. So I guess if you wanted to try this, maybe you could set up three scripts, one set to 80% of your bandwidth, one set to 50%, and one set to 20%. Then set up a cron job that checks the "ping" for your high priority traffic, and calls the appropriate script to adjust. I doubt it''ll work, but it could be an interesting experiment. ----- Original Message ----- From: "Justin Schoeman" <justin@expertron.co.za> To: <lartc@mailman.ds9a.nl> Sent: Thursday, November 25, 2004 3:17 AM Subject: [LARTC] Shaping traffic on heavily oversubscribed links?> Hi all, > > I am having some fun with traffic shaping, and have run into an > interesting situation. Here is South Africa, most internet links are > heavily oversubscribed, which means that in most cases the local link is > _not_ the bottleneck, and shaping on the local link does not help that > much... > > Does anybody have some tips on shaping such links? How can you get > interractive traffic if you don''t know how much bandwidth to reserve for > it? How can you give fair access to a link if you don''t know what the link > capacity is? > > Are there perhaps some tools to monitor retransmissions to try and > determine congestion levels, and from that adjust shaped bandwidth? > > Am I perhaps missing something simple in this scenario? > > Thanks! > -justin > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
it''s worse than that. we faced the same problems in china. an oversubscribed adsl system. in fact there is more than one problem with the public internet in these scenarios. the first as you have identified is the lack of capacity. then you can''t control the downstream routing - eg traffic from china to australia was going to the west coast us, then to florida and finally back to australia via a different gateway in la! some of the routers along the way have large buffers meaning that they can introduce significant delays into the transmission. and finally when all these things overflow you get large packet loss - up to 50% at some times. the net result is this: high packet loss means forget about real time services such as voip or video conferencing. they are asynchronous services that don''t retransmit and are also not very tolerant of packet loss (over about 10%). long ping times means that synchronous services such as email are very slow, coupled with packet loss you might as well be on a dial up. our solution in a commercial environment was to keep upgrading until we found a service level that works (more or less). we moved from adsl to fibre. found an isp at each end that we could talk to and who could give assistance with routing issues. eg if they have multiple gateways we experimented until we found the most efficient gateway. we also know of larger comapnies who pay substantial fees to achieve end to end qos. all of this means there''s not a lot you can do at your end with limited choices except to accept that there is no bandwidth at some times of the day and go have a beer instead :) regards rick Chris Bennett wrote:> Quick answer is: you can''t. You need to know the bandwidth so that > you can control the queue. > > You bring up some interesting theoretical ideas about monitoring the > overall congestion level and forcing back-offs when you sense that > you''re reaching the current level, though. That would sort of involve > dynamically adjusting the shaping rules. As far as I know there is > nothing in existance that does this, but its an interesting idea to > think about. If you were going to try something like this I think > you''d need some sort of reliable indicator of what the current > congestion is like... perhaps some steady ping to use as your "canary > in the coal mine". Then set up several shaping scripts that assume > different levels of bandwidth, and depending on the current "canary > ping", either upgrade or downgrade your assumed bandwidth by calling > the appropriate script. So I guess if you wanted to try this, maybe > you could set up three scripts, one set to 80% of your bandwidth, one > set to 50%, and one set to 20%. Then set up a cron job that checks > the "ping" for your high priority traffic, and calls the appropriate > script to adjust. I doubt it''ll work, but it could be an interesting > experiment. > > ----- Original Message ----- From: "Justin Schoeman" > <justin@expertron.co.za> > To: <lartc@mailman.ds9a.nl> > Sent: Thursday, November 25, 2004 3:17 AM > Subject: [LARTC] Shaping traffic on heavily oversubscribed links? > > >> Hi all, >> >> I am having some fun with traffic shaping, and have run into an >> interesting situation. Here is South Africa, most internet links are >> heavily oversubscribed, which means that in most cases the local link >> is _not_ the bottleneck, and shaping on the local link does not help >> that much... >> >> Does anybody have some tips on shaping such links? How can you get >> interractive traffic if you don''t know how much bandwidth to reserve >> for it? How can you give fair access to a link if you don''t know what >> the link capacity is? >> >> Are there perhaps some tools to monitor retransmissions to try and >> determine congestion levels, and from that adjust shaped bandwidth? >> >> Am I perhaps missing something simple in this scenario? >> >> Thanks! >> -justin >> _______________________________________________ >> LARTC mailing list / LARTC@mailman.ds9a.nl >> http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ >> > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Justin Schoeman
2004-Nov-26 09:00 UTC
Re: Shaping traffic on heavily oversubscribed links?
Thanks everybody for your advice... This is going to be an interesting one to try and solve ;-). -justin Justin Schoeman wrote:> Hi all, > > I am having some fun with traffic shaping, and have run into an > interesting situation. Here is South Africa, most internet links are > heavily oversubscribed, which means that in most cases the local link is > _not_ the bottleneck, and shaping on the local link does not help that > much... > > Does anybody have some tips on shaping such links? How can you get > interractive traffic if you don''t know how much bandwidth to reserve for > it? How can you give fair access to a link if you don''t know what the > link capacity is? > > Are there perhaps some tools to monitor retransmissions to try and > determine congestion levels, and from that adjust shaped bandwidth? > > Am I perhaps missing something simple in this scenario? > > Thanks! > -justin > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
On Thursday 25 November 2004 13:01, Chris Bennett wrote:> Quick answer is: you can''t. You need to know the bandwidth so that you can > control the queue.Indeed. I suffer from the same problem with my PPPoATM link, where my shaping configuration assumes I''m operating over an Ethernet link when it''s really encapsulated over an ATM network. Bandwidth available for Ethernet packets varies depending on how many packets are going out over the link and the size of each packet. -- Jason Boxman Perl Programmer / *NIX Systems Administrator Shimberg Center for Affordable Housing | University of Florida http://edseek.com/ - Linux and FOSS stuff _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Dimitris Kotsonis
2004-Dec-06 11:54 UTC
Re: Shaping traffic on heavily oversubscribed links?
Justin Schoeman wrote:> Hi all, > > I am having some fun with traffic shaping, and have run into an > interesting situation. Here is South Africa, most internet links are > heavily oversubscribed, which means that in most cases the local link > is _not_ the bottleneck, and shaping on the local link does not help > that much... >We have the same problem in DSL lines here in Greece. I have found that while the average efective speed on such lines varies, tha average rate of packets is more or less constant. I have a theory for this. I believe that the routers that forward the traffic on congested lines - on ISPs and on the ATM circuits at the telcoms - don''t take the extra time needed to calculate the size of the packets and distribute the traffic on a per packet basis. This leads to a ''fairness'' among the end receivers based on packets/sec instead of bandwidth. To be more specific. In my ADSL line I usually achieve between 20-30 pps (measured with MRTG). With an average packet size of 1500 this is 20-45 kbytes/sec. But packets sizes close to the MTU are found on single ftp/http connections and pretty much nowehere else. Packet sizes of 400 to 500 are more realistic, especially when p2p programs are involved. 20-30 packets give 8-10kbytes/sec. You can expect even less when using voip programs which utilize smaller packets. If you find that single a FTP session tends to get more bandwidth thatn p2p programs or multiuser traffic then you have a simillar problem to our own. I would suggest that you setup MRTG to monitor packets to research further into this.> Does anybody have some tips on shaping such links? How can you get > interractive traffic if you don''t know how much bandwidth to reserve > for it? How can you give fair access to a link if you don''t know what > the link capacity is? >Well, I am working on one. Since I can''t shape bandwidth because it flactuates erratically with time and usage I decided to shape packets. I have created a new queueing discipline based on TBF which uses packets instead of bytes for its tokens and I am allocating a constant packet/sec rate on each user of my ADSL line. A better solution would be to create an HTB alike packet-based qdisc for dynamic shaping. If you find that you have the some problem as me and you want to experiment with a packet-based TBF qdisc I can send you a patch for linux-2.6.8 and iproute2 in this list. I would like to here your thought on this anyway ... Dimitris _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Dimitris Kotsonis wrote:> Justin Schoeman wrote: > >> Hi all, >> >> I am having some fun with traffic shaping, and have run into an >> interesting situation. Here is South Africa, most internet links are >> heavily oversubscribed, which means that in most cases the local link >> is _not_ the bottleneck, and shaping on the local link does not help >> that much... >> > > We have the same problem in DSL lines here in Greece. > > I have found that while the average efective speed on such lines varies, > tha average rate of packets is more or less constant. I have a theory > for this. I believe that the routers that forward the traffic on > congested lines - on ISPs and on the ATM circuits at the telcoms - don''t > take the extra time needed to calculate the size of the packets and > distribute the traffic on a per packet basis. This leads to a ''fairness'' > among the end receivers based on packets/sec instead of bandwidth. > > To be more specific. In my ADSL line I usually achieve between 20-30 pps > (measured with MRTG). With an average packet size of 1500 this is 20-45 > kbytes/sec. But packets sizes close to the MTU are found on single > ftp/http connections and pretty much nowehere else. Packet sizes of 400 > to 500 are more realistic, especially when p2p programs are involved. > 20-30 packets give 8-10kbytes/sec. You can expect even less when using > voip programs which utilize smaller packets. > > If you find that single a FTP session tends to get more bandwidth thatn > p2p programs or multiuser traffic then you have a simillar problem to > our own. I would suggest that you setup MRTG to monitor packets to > research further into this.It is normal for an FTP download to take over from p2ps the latter are likely to be higher latency, so TCP will let a lower latency FTP grab more bandwidth. Try shaping with HTB and sfq - It should help.> > > >> Does anybody have some tips on shaping such links? How can you get >> interractive traffic if you don''t know how much bandwidth to reserve >> for it? How can you give fair access to a link if you don''t know what >> the link capacity is? >> > > Well, I am working on one. Since I can''t shape bandwidth because it > flactuates erratically with time and usage I decided to shape packets. I > have created a new queueing discipline based on TBF which uses packets > instead of bytes for its tokens and I am allocating a constant > packet/sec rate on each user of my ADSL line. A better solution would be > to create an HTB alike packet-based qdisc for dynamic shaping. > > If you find that you have the some problem as me and you want to > experiment with a packet-based TBF qdisc I can send you a patch for > linux-2.6.8 and iproute2 in this list. > > I would like to here your thought on this anyway ... > > Dimitris > > > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/