Hi,
at my local university network, I have to make sure no student
uploads more than x GB/day. So far, I give them unlimited bandwidth
until they have more than y GB (y<x) upload. After that, I use the
u32 filter to associate the affected IP address with a HTB class
limited to the rate (remainingtraffic/remainingtime).
Since the accounting is done with ipt_ACCOUNT right now and the
netfilter framework obviously can''t know whether a packet was
dropped because the queue was full, dropped packets are counted as
traffic, but that is somewhat unfair.
Can I do the accounting with the traffic control framework in the
kernel instead of using ipt_ACCOUNT? Giving every IP its own htb
class and setting that to an incredibly high limit to do the
accounting seems overkill (there is no point in forcing HTB to
do all the calculations for shaping if I only want to count)
especially because I''d have to do that for a sparsely populated
/16 network (about 1500-2800 hosts).
One (not that sophisticted) idea is to leave the accounting "as is"
and add a "dropped bytes" count to htb. This could be subtracted
from the numbers ipt_ACCOUNT gives me. That would modify the output
of "tc class ls" from
Sent 86136037 bytes 103963 pkt (dropped 283998, overlimits 0 requeues 0)
to
Sent 86136037 bytes 103963 pkt (dropped 385953282 bytes 283998 pkt, overlimits 0
requeues 0)
Proposal:
----------------------------------
Another idea would be to create a qdisc HTBQ (HTB with quota)
derived from HTB with the following characteristics:
htb_rate=min(htbq_rate,
(alreadysent=>htbq_squota)?((htbq_quota-alreadysent)/remtime):htbq_rate)
htb_ceil=htbq_ceil //this is just passed on
htb_burst=htbq_burst
htb_cburst=htbq_cburst
htb_prio=htbq_prio
htb_quantum=htbq_quantum //should be set automatically
htbq_interval seconds //time after which quota is reset
htbq_starttime seconds //time when the first interval starts
htbq_quota bytes //maximum allowed bytes
htbq_squota bytes //unshaped quota
htbq_rate is optional, unlimited if not set
htbq_ceil is optional
htbq_burst is optional
htbq_cburst is optional
htbq_prio is mandantory
htbq_quantum is optional
htbq_interval is mandantory
htbq_starttime is optional, defaults to unixtime 0
htbq_quota is mandantory
htbq_squota is optional, defaults to 0 (shape from beginning)
Basic description:
Case 1: already sent bytes are less than htbq_squota
if htbq_{rate,ceil,burst,cburst,quantum} is set then
do normal htb shaping with copied parameters
else
pass on packets directly to network device
Case 2: already sent bytes are greater or equal than htbq_squota
if htbq_{rate,ceil,burst,cburst,quantum} is set then
do htb shaping with copied parameters except
htb_rate=min(htbq_rate, (htbq_quota-alreadysent)/remtime)
htb_ceil=min(htbq_ceil, (htbq_quota-alreadysent)/remtime)
else
do htb shaping with following parameters
htb_rate=(htbq_quota-alreadysent)/remtime
htb_ceil=(htbq_quota-alreadysent)/remtime
This would surely be helpful for some admins who have to limit
users to a certain quota without constantly shaping their
network traffic or pulling the plug once the quota is full.
Thoughts?
Regards,
Carl-Daniel
--
http://www.hailfinger.org/
On Sat, 04 Jun 2005 14:45:30 +0200 Carl-Daniel Hailfinger <c-d.hailfinger.devel.2005@gmx.net> wrote:>Hi,Hi.>Since the accounting is done with ipt_ACCOUNT right now and the >netfilter framework obviously can''t know whether a packet was >dropped because the queue was full, dropped packets are counted as >traffic, but that is somewhat unfair.I did some tests about a year ago and arrived to the conclusion that this difference can be ignored for practical purposes (~2%). Yours sincerely, Peter
Hi, Peter Surda schrieb:> On Sat, 04 Jun 2005 14:45:30 +0200 Carl-Daniel Hailfinger > >>Since the accounting is done with ipt_ACCOUNT right now and the >>netfilter framework obviously can''t know whether a packet was >>dropped because the queue was full, dropped packets are counted as >>traffic, but that is somewhat unfair. > > I did some tests about a year ago and arrived to the conclusion that this > difference can be ignored for practical purposes (~2%).I know that it can be ignored most of the time, but certain types of communication using UDP packets don''t adjust their rates like a TCP connection would. One pathological case was >70% dropped packets and the netfilter counters were complete garbage. Regards, Carl-Daniel -- http://www.hailfinger.org/
Carl-Daniel Hailfinger wrote:> Proposal: > ---------------------------------- > Another idea would be to create a qdisc HTBQ (HTB with quota) > derived from HTB with the following characteristics: > > htb_rate=min(htbq_rate, (alreadysent=>htbq_squota)?((htbq_quota-alreadysent)/remtime):htbq_rate)Why do you want to decrease speed as the quota is approached? Wouldn''t a simple per-class quota or a quota-ematch work as well? Regards Patrick
On Sat, 04 Jun 2005 15:21:36 +0200 Carl-Daniel Hailfinger <c-d.hailfinger.devel.2005@gmx.net> wrote:>Hi,Hi,>I know that it can be ignored most of the time, but certain types >of communication using UDP packets don''t adjust their rates like >a TCP connection would. One pathological case was >70% dropped >packets and the netfilter counters were complete garbage.In that case, on the same website as wrr ( http://wipl-wrr.sf.net/ ), there is another program, wipl. It is supposed to do IP-accounting based on tc (I don''t know if it depends on wrr, I never used it, it just says it was designed for the purpose you require).>Regards, >Carl-DanielYours sincerely, Peter
Patrick McHardy schrieb:> Carl-Daniel Hailfinger wrote: > >>Proposal: >>---------------------------------- >>Another idea would be to create a qdisc HTBQ (HTB with quota) >>derived from HTB with the following characteristics: >> >>htb_rate=min(htbq_rate, (alreadysent=>htbq_squota)?((htbq_quota-alreadysent)/remtime):htbq_rate) > > > Why do you want to decrease speed as the quota is approached?We have two phases (simplified): 1. Already sent traffic is less than htbq_squota -> Do not limit anything. 2. Already sent traffic is more than htbq_squota -> Limit the rate to something which allows the quota to be filled completely in the remainig time. Most people stay below htbq_squota and do not notice anything at all, i.e. they get full wire speed. Power users will cause more traffic than htb_squota and be limited so they can''t get over htbq_quota. Since it is impossible to send faster than htb_rate, htb_rate will stay constant if it is fully utilized. If htb_rate is not fully utilized, the speed will actually *increase* over time.> Wouldn''t a simple per-class quota or a quota-ematch work as > well?That would be easier, but I can''t see how to realize the process above with a quota-ematch. Especially the dynamic rate adaption needs something more than just quota. Probably one could combine quota-ematch with some userspace hackery to achieve what I want, but it would require to statically set up 65536 classes for a /16 network. Hm. The only difference to my solution would be that we need a quota ematch instead of a htb_quota qdisc and rate control would be done in userspace. So where can I find code doing a quota ematch? Regards, Carl-Daniel -- http://www.hailfinger.org/
Carl-Daniel Hailfinger wrote:> Patrick McHardy schrieb: > >>Why do you want to decrease speed as the quota is approached? > > > We have two phases (simplified): > 1. Already sent traffic is less than htbq_squota > -> Do not limit anything. > 2. Already sent traffic is more than htbq_squota > -> Limit the rate to something which allows the quota > to be filled completely in the remainig time. > > Most people stay below htbq_squota and do not notice > anything at all, i.e. they get full wire speed. Power users > will cause more traffic than htb_squota and be limited so > they can''t get over htbq_quota. Since it is impossible to > send faster than htb_rate, htb_rate will stay constant if > it is fully utilized. If htb_rate is not fully utilized, > the speed will actually *increase* over time.Ok, so why do you want to prevent people from sending from htbq_squota to htbq_quota at full speed? Isn''t the important point that noone sends more than their quota? Dynamic rate adaption is not easy, HTB needs pre-calculated rate tables, with HFSC it is even more complicated and can cause short periods of unfairness.>>Wouldn''t a simple per-class quota or a quota-ematch work as >>well? > > That would be easier, but I can''t see how to realize the > process above with a quota-ematch. Especially the dynamic > rate adaption needs something more than just quota. > Probably one could combine quota-ematch with some userspace > hackery to achieve what I want, but it would require to > statically set up 65536 classes for a /16 network. Hm. > The only difference to my solution would be that we need > a quota ematch instead of a htb_quota qdisc and rate control > would be done in userspace. > > So where can I find code doing a quota ematch?Nowhere I guess, but shouldn''t take more than a couple of minutes to write one. It would have the same problems as with iptables however, it doesn''t get feedback about dropped packets. A per-class quota inside the qdisc is probably the best way. Regards Patrick