Hello List, I''m new to QoS/tcng/HTB and friends, so please forgive me if my question might be silly... After having read lots of HowTo documents I''m totally confused... The Challenge: =============I''ll have to deploy several "mirror" download servers (Linux) which must be able to handle a huge number of HTTP download requests (about 10k to 20k unicast requests per server within 120 min.) which will hit my servers . The mirror servers will be connected with 100MBit to the Internet backbone. Each "client" will download between 7MB up to 30MB per request. The download mirror servers will be directly attached to the Internet (no FW or router between, which could handle QoS for me). Of course will these download mirrors be specially hardened :-) The Idea: ========I can not handle all the traffic at once without QoS, so I thought it should be possible to establish a traffic control mechanism to guarantee bandwith for each client download (e.g. 256kbps CIR up to 512kbps ceil). This should allow me, to "serve" all clients within 120 min, before a new download wave will hit my servers. If the available bandwith for HTTP traffic is used, clients shall not be able to start a download, until some bandwith is freed by another client who finished a download. Note: I would rather go with "tcng" than with the old "tc". The Problem: ===========While playing with tcng and HTB I managed to shape traffic (to limit the total bandwith for HTTP traffic to e.g. 50Mbps) but I did not manage to write a tcng configuration which guarantees a certain bandwith per client request. I must be able to assign (and guarantee) a defined bandwith to a client request, to make the downloads more predictable. My experiments so far only allowed me to limit e.g. HTTP traffic, which then was equally distributed to the number of clients, but this led to the usual "traffic jam" when too many clients initiated a simultanes download. Can anybody please assist me, or does anybody have a working "tcng" config file to do the job. Additional readings/comments/links (beside those listed in the LDB/Linux Traffic Control Howtos are also welcome.) Many thanks in advance and best regards from Austria Thomas
On Mer 27 avril 2005 18:11, Thomas Mandl a écrit :> Hello List, > I''m new to QoS/tcng/HTB and friends, so please forgive me if my questionmight be silly...> After having read lots of HowTo documents I''m totally confused... > > The Challenge: > =============> I''ll have to deploy several "mirror" download servers (Linux) which must be > able to handle a huge number of HTTP download requests (about 10k to 20kunicast requests per server within 120 min.) which will hit my servers . The> mirror servers will be connected with 100MBit to the Internet backbone.Each> "client" will download between 7MB up to 30MB per request. The downloadmirror servers will be directly attached to the Internet (no FW or router between, which could handle QoS for me). Of course will these download mirrors be specially hardened :-)> > The Idea: > ========> I can not handle all the traffic at once without QoS, so I thought itshould> be possible to establish a traffic control mechanism to guaranteebandwith for each client download (e.g. 256kbps CIR up to 512kbps ceil). This should> allow me, to "serve" all clients within 120 min, before a new download wave > will hit my servers. If the available bandwith for HTTP traffic is used,clients shall not be able to start a download, until some bandwith is freed> by another client who finished a download. > > Note: I would rather go with "tcng" than with the old "tc". > > The Problem: > ===========> While playing with tcng and HTB I managed to shape traffic (to limit thetotal bandwith for HTTP traffic to e.g. 50Mbps) but I did not manage to write a tcng configuration which guarantees a certain bandwith per client request. I must be able to assign (and guarantee) a defined bandwith to a client request, to make the downloads more predictable. My experiments so far only allowed me to limit e.g. HTTP traffic, which then was equally distributed to the number of clients, but this led to the usual "traffic jam" when too many clients initiated a simultanes download.> > Can anybody please assist me, or does anybody have a working "tcng"config file to do the job. Additional readings/comments/links (beside those listed> in the LDB/Linux Traffic Control Howtos are also welcome.) > > Many thanks in advance and > best regards from Austria > > Thomas >I think you might want to check Apache''s mod_bandwidth / mod_throttle instead of QoS. Regards, Sylvain
On Wednesday 27 April 2005 12:17, Sylvain BERTRAND wrote: <snip>> I think you might want to check Apache''s mod_bandwidth / mod_throttle > instead of QoS.There''s also bw_mod if you''re running Apache2, which for me is working fine so far. http://www.ivn.cl/apache/
Thomas Mandl wrote:> Hello List, > I''m new to QoS/tcng/HTB and friends, so please forgive me if my question > might be silly... > After having read lots of HowTo documents I''m totally confused... > > The Challenge: > =============> I''ll have to deploy several "mirror" download servers (Linux) which must be > able to handle a huge number of HTTP download requests (about 10k to 20k > unicast requests per server within 120 min.) which will hit my servers . The > mirror servers will be connected with 100MBit to the Internet backbone. Each > "client" will download between 7MB up to 30MB per request. The download > mirror servers will be directly attached to the Internet (no FW or router > between, which could handle QoS for me). Of course will these download > mirrors be specially hardened :-) > > The Idea: > ========> I can not handle all the traffic at once without QoS, so I thought it should > be possible to establish a traffic control mechanism to guarantee bandwith > for each client download (e.g. 256kbps CIR up to 512kbps ceil). This should > allow me, to "serve" all clients within 120 min, before a new download wave > will hit my servers. If the available bandwith for HTTP traffic is used, > clients shall not be able to start a download, until some bandwith is freed > by another client who finished a download. > > Note: I would rather go with "tcng" than with the old "tc". > > The Problem: > ===========> While playing with tcng and HTB I managed to shape traffic (to limit the > total bandwith for HTTP traffic to e.g. 50Mbps) but I did not manage to > write a tcng configuration which guarantees a certain bandwith per client > request. I must be able to assign (and guarantee) a defined bandwith to a > client request, to make the downloads more predictable. My experiments so > far only allowed me to limit e.g. HTTP traffic, which then was equally > distributed to the number of clients, but this led to the usual "traffic > jam" when too many clients initiated a simultanes download. > > Can anybody please assist me, or does anybody have a working "tcng" config > file to do the job. Additional readings/comments/links (beside those listed > in the LDB/Linux Traffic Control Howtos are also welcome.)Wang Jian posted a perflow queue recently - http://mailman.ds9a.nl/pipermail/lartc/2005q2/015381.html It is the sort of thing you need if you can''t do it with apache as recommended. Andy.
Hello Everybody, I forgot to mention, that my HTTP traffic will not be handled by an apache web server, and the HTTP protocol can be replaced in the near future by a different protocol. Thus apache''s mod_bandwith isn''t an option for me right now. I''m not sure how Wang Jian''s post can help me. I thought the existing Linux 2.6.11 kernel and tcng already can do what I need? All I have to do is write a proper tcng config file... But the proper tcng file is the challenge... regards Thomas -----Ursprüngliche Nachricht----- Von: Andy Furniss [mailto:andy.furniss@dsl.pipex.com] Gesendet: Mittwoch, 27. April 2005 23:23 An: mandl.t@ikarus.at Cc: LARTC Betreff: Re: [LARTC] urgent question about tcng! Thomas Mandl wrote:> Hello List, > I''m new to QoS/tcng/HTB and friends, so please forgive me if my > question might be silly... > After having read lots of HowTo documents I''m totally confused... > > The Challenge: > =============> I''ll have to deploy several "mirror" download servers (Linux) which > must be able to handle a huge number of HTTP download requests (about > 10k to 20k unicast requests per server within 120 min.) which will hit > my servers . The mirror servers will be connected with 100MBit to the > Internet backbone. Each "client" will download between 7MB up to 30MB > per request. The download mirror servers will be directly attached to > the Internet (no FW or router between, which could handle QoS for me). > Of course will these download mirrors be specially hardened :-)[...snip...]> > Can anybody please assist me, or does anybody have a working "tcng" > config file to do the job. Additional readings/comments/links (beside > those listed in the LDB/Linux Traffic Control Howtos are also > welcome.)Wang Jian posted a perflow queue recently - http://mailman.ds9a.nl/pipermail/lartc/2005q2/015381.html It is the sort of thing you need if you can''t do it with apache as recommended. Andy.
Thomas Mandl wrote:> Hello Everybody, > > I forgot to mention, that my HTTP traffic will not be handled by an apache > web server, and the HTTP protocol can be replaced in the near future by a > different protocol. Thus apache''s mod_bandwith isn''t an option for me right > now. > > I''m not sure how Wang Jian''s post can help me. I thought the existing Linux > 2.6.11 kernel and tcng already can do what I need? All I have to do is write > a proper tcng config file... > > But the proper tcng file is the challenge...I don''t know tcng, but the reason I suggested perflow is that you want each flow to have a ceil - unless you make a class and rule to match each flow I can''t see how you can do this. Also iptables could limit the number of connections - tc can''t, perflow can. Andy.
Andy Furniss wrote:> > I don''t know tcng, but the reason I suggested perflow is that you want > each flow to have a ceil - unless you make a class and rule to match > each flow I can''t see how you can do this. Also iptables could limit the > number of connections - tc can''t, perflow can.Forgot to say you can also use iptables to limit rate per connection. Andy.
Sorry for the stupid question, but how would I rate limit connections using iptables? regards Thomas -----Ursprüngliche Nachricht----- Von: Andy Furniss [mailto:andy.furniss@dsl.pipex.com] Gesendet: Dienstag, 03. Mai 2005 16:43 An: Andy Furniss Cc: mandl.t@ikarus.at; ''LARTC'' Betreff: Re: AW: [LARTC] urgent question about tcng! Andy Furniss wrote:> > I don''t know tcng, but the reason I suggested perflow is that you want > each flow to have a ceil - unless you make a class and rule to match > each flow I can''t see how you can do this. Also iptables could limit > the number of connections - tc can''t, perflow can.Forgot to say you can also use iptables to limit rate per connection. Andy.
Thomas Mandl wrote:> Sorry for the stupid question, but how would I rate limit connections using > iptables? >I was thinking of the patch-o-matic-ng patches connlimit and connrate, though I''ve never used either and they may or may not be expensive for many connections compared to perflow. Connrate lets you mark packets if they are above limits - you could then drop those later in a filter table or with a TC filter/queue.> regards > Thomas > -----Ursprüngliche Nachricht----- > Von: Andy Furniss [mailto:andy.furniss@dsl.pipex.com] > Gesendet: Dienstag, 03. Mai 2005 16:43 > An: Andy Furniss > Cc: mandl.t@ikarus.at; ''LARTC'' > Betreff: Re: AW: [LARTC] urgent question about tcng! > > Andy Furniss wrote: > > >>I don''t know tcng, but the reason I suggested perflow is that you want >>each flow to have a ceil - unless you make a class and rule to match >>each flow I can''t see how you can do this. Also iptables could limit >>the number of connections - tc can''t, perflow can. > > > Forgot to say you can also use iptables to limit rate per connection. > > Andy. > >