First I''ll confess my sins, then I''ll beg for help.
I own a small wireless ISP, and I sell service at three levels, 256kb/s,
384kb/s, and 512kb/s. For about 18 months, I thought I had this bandwidth
limiting figured out. I had three HTB classes off the root, one each with
the limits above. Since each customer has a single IP address, I used
iptables to mark packets destined for each subscriber with level 1,2 or 3.
It looked like it worked great: Customer A got 256, B got 384, and C got his
512. In hindsight, it was wishful thinking, since ALL 256kb/s customers got
dumped into the same 256 class and had to duke it out for that 256kb/s,
instead of each customer getting their own 256kb/s slice. Eventually, as I
added more customers, people started complaining that they weren''t
getting
what they''re paying for (rightly).
So, now I''m running my ISP with no bandwidth shaping and I''m
struggling to get
my brain wrapped around how make sure everyone gets what they pay for, but
not more.
From the reading I''ve done, it looks like I need a separate class for
each
subscriber. Inside that class, I''d like to have a standard set of
queues to
prioritize each customer''s slice of bandwidth by port (typical three
band
stuff: interactive, web,bulk).
So assuming I''ve got three customers:
10.0.0.10 gets256kb/s
10.0.0.11 gets 386 kb/s
10.0.0.12 gets 256kb/s
I think I''d have a tree like this:
CLASS 10 (256kb/s) (inner classes prioritize)
ROOT ----- CLASS 11(384kb/s) "
CLASS 12 (256kb/s) "
plus three filters to direct iptables-marked traffic to the appropriate queue
and three iptables entries to mark the traffic appropriately.
The problem is, I''m simply not smart enough to actually IMPLEMENT this.
I
tried setting up the HTB classes, and when I added the 5th one (class 15 in
the little diagram above), my interactivity to the router went to near 1000ms
RTT (I was ssh''ed into it). I started getting calls from my customers
immediately saying something was ''wrong with the internet''.
I''m not sure
what happened, since I hadn''t installed any filters or anything.
Anyway, I''d dearly appreciate some help on this. Surely this is a nut
that
someone has already cracked, but for the life of me all I can find on the
internet are how-to shape your own outbound traffic to your ISP so your P2P
traffic (the BANE of ISPs) doesn''t interfere with your Doom3
deathmatch.
Thanks in advance,
John Gorkos
I also run a small wireless ISP, and we use HTB for shaping. You need a separate class for each customer. Alternatively I think it might be ok to use SFQ with multiple customers in the same class, but for us the class per customer works fine. Fundamentally you need some mechanism that ensures that each customer''s packets are queued in their own queue. Otherwise they will interfere with each other''s QoS.
John, Personally I think you''re doing too much. All the ISPs here offer a specified amount of bandwidth to different classes of customer. However, they also list a contention ratio. Thus they might say customer class A gets 512Kbit/s with a contention rate of 8:1 and customer class B gets 256Kbit/s with a contention ratio of 12:1. This is very simple to set up. Assume we have eight class A customers and twelve class B customers. Similar to what you did before, set up two HTB classes off the root and use ip tables to mark customer packets according to their assigned IP address. Make sure that the HTB classes are using SFQ. Now all eight class A customers will share the 512Kbit/s class, with traffic distributed evenly among all active flows. If this type of solution is not satisfactory for your customers then you will need to set up a separate class for each individual customer and mark their packets appropriately. This may be more hassle than its worth, depending on the number of customers you have. Regards Brian Carrig On 27 Apr 2005 at 10:35, John Gorkos wrote:> First I''ll confess my sins, then I''ll beg for help. > > I own a small wireless ISP, and I sell service at three levels, > 256kb/s, 384kb/s, and 512kb/s. For about 18 months, I thought I had > this bandwidth limiting figured out. I had three HTB classes off the > root, one each with the limits above. Since each customer has a > single IP address, I used iptables to mark packets destined for each > subscriber with level 1,2 or 3. It looked like it worked great: > Customer A got 256, B got 384, and C got his 512. In hindsight, it > was wishful thinking, since ALL 256kb/s customers got dumped into the > same 256 class and had to duke it out for that 256kb/s, instead of > each customer getting their own 256kb/s slice. Eventually, as I added > more customers, people started complaining that they weren''t getting > what they''re paying for (rightly). > > So, now I''m running my ISP with no bandwidth shaping and I''m > struggling to get my brain wrapped around how make sure everyone gets > what they pay for, but not more. > > From the reading I''ve done, it looks like I need a separate class for > each subscriber. Inside that class, I''d like to have a standard set > of queues to prioritize each customer''s slice of bandwidth by port > (typical three band stuff: interactive, web,bulk). > So assuming I''ve got three customers: > 10.0.0.10 gets256kb/s > 10.0.0.11 gets 386 kb/s > 10.0.0.12 gets 256kb/s > I think I''d have a tree like this: > > CLASS 10 (256kb/s) (inner classes prioritize) > ROOT ----- CLASS 11(384kb/s) " > CLASS 12 (256kb/s) " > plus three filters to direct iptables-marked traffic to the > appropriate queue and three iptables entries to mark the traffic > appropriately. > > The problem is, I''m simply not smart enough to actually IMPLEMENT > this. I tried setting up the HTB classes, and when I added the 5th > one (class 15 in the little diagram above), my interactivity to the > router went to near 1000ms RTT (I was ssh''ed into it). I started > getting calls from my customers immediately saying something was > ''wrong with the internet''. I''m not sure what happened, since I hadn''t > installed any filters or anything. > > Anyway, I''d dearly appreciate some help on this. Surely this is a nut > that someone has already cracked, but for the life of me all I can > find on the internet are how-to shape your own outbound traffic to > your ISP so your P2P traffic (the BANE of ISPs) doesn''t interfere with > your Doom3 deathmatch. > > Thanks in advance, > John Gorkos > _______________________________________________ > LARTC mailing list > LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc-- Brian Carrig Research Assistant Department of Computing & Networking Institute of Technology, Carlow Tel. No.: +353 59 9176314
On Wednesday 27 April 2005 17:35, John Gorkos wrote:> Anyway, I''d dearly appreciate some help on this. Surely this is a nut > that someone has already cracked, but for the life of me all I can find > on the internet are how-to shape your own outbound traffic to your ISPI may be really close to this one. Although I''m not an ISP and don''t have customers per se, I do have flatmates and a router, and I''m pretty much using the one class per user approach you''re describing. Although I know of someone who is using a modified version of this script in a wireless network, it''s developed with ''home router'' (especially my own router) and ''fair sharing'' in mind. So it has many downsides for you; it kicks out your firewall, works only with static IPs, and capping users to 256/368/512kbit will only affect ceil rate. On the bright side, as long as you have one device for internet, one device for customers, and your customers have static IPs on this device, it might work out of the box for you, and even if it doesn''t, it comes with some documentation and structure that is supposed to make it easy to understand, extract ideas and modify. And from what I read in the feedback I get, most people use it this way. ;-) You can find my script here: http://www.metamorpher.de/fairnat HTH Andreas
On Wednesday 27 April 2005 13:42, Andreas Klauer wrote:> On Wednesday 27 April 2005 17:35, John Gorkos wrote: > > Anyway, I''d dearly appreciate some help on this. Surely this is a nut > > that someone has already cracked, but for the life of me all I can find > > on the internet are how-to shape your own outbound traffic to your ISP > > I may be really close to this one. Although I''m not an ISP and don''t have > customers per se, I do have flatmates and a router, and I''m pretty much > using the one class per user approach you''re describing. > > Although I know of someone who is using a modified version of this script > in a wireless network, it''s developed with ''home router'' (especially my > own router) and ''fair sharing'' in mind. So it has many downsides for you; > it kicks out your firewall, works only with static IPs, and capping users > to 256/368/512kbit will only affect ceil rate. > > On the bright side, as long as you have one device for internet, one device > for customers, and your customers have static IPs on this device, it might > work out of the box for you, and even if it doesn''t, it comes with some > documentation and structure that is supposed to make it easy to > understand, extract ideas and modify. And from what I read in the feedback > I get, most people use it this way. ;-) > > You can find my script here: http://www.metamorpher.de/fairnat > > HTH > AndreasThis is remarkably close to what I''m looking for. The grin on my face should be visible from Germany. I have a question about the following section from the config file: # --- Clients: --- # Specify the clients for which we do Masquerading and Shaping. # The script assumes that all clients have static IPs in the # same subnet as your LAN device. # # Example: If the IP of DEV_LAN is 192.168.100.42, the line above # means that 192.168.100.2, 192.168.100.5, etc., are # the IPs of your clients. # 6:23:25 is a group of 3 IPs that all belong to the same user. # Use this notation if a single person has more than one machine # /IP connected to the router. # # New: You can also specify a custom ceil rate per user. # Syntax: <user>@<down_ceil>[|<up_ceil>] # However, ceil has to be bigger than the guaranteed rate, # otherwise you will get weird results. USERS="2 5 6:23:25 183@1mbit|100kbit" How do I include multiple lines of USERS= here. For example, users with IP addresss 11,12,13,14,15 are all 256kb/s users, 16,17,18,19 are 384kb/s users, and 20-30 are back to 256kb/s users. Is this acceptable: USERS="\ 11 12 13 14 15 183@256kbit/256kbit\ 16 17 18 19 184@384kbit/384kbit\ 20 21 22 23 24 25 26 27 28 29 30 185@256kbit/256kbit" ? Also, what does the 183 (184,185 in my example) signify? Thanks tons! John Gorkos
On Thursday 28 April 2005 00:19, you wrote:> # Syntax: <user>@<down_ceil>[|<up_ceil>] > How do I include multiple lines of USERS= here.Maybe the example is bad. USERS does not have lines.> Also, what does the 183 (184,185 in my example) signify?The same as 20, 21, 22, etc. in your example. That''s one user. You''ll have to specify ceil rates for all users you want to be limited that way. Also notice that it''s | instead of /. For example, 20@512kbit|256kbit translates to: User 20 will be ceiled at 512kbit download and 256kbit upload. The number 20 will be expanded to the IP of that user in your subnet, so if your subnet is 192.168.100.x, the IP will be 192.168.100.20. You can also group multiple IPs together, so 20:21:22@256kbit|256kbit would be similar to your old setup, throwing traffic of all IPs together in the same class and let them battle it out. So if I interpret your example right, it should look like this: USERS="11@256kbit|256kbit 12@256kbit|256kbit 13@256kbit|256kbit 14@256kbit|256kbit 15@256kbit|256kbit 16@386kbit|386kbit 17@386kbit|386kbit 18@386kbit|386kbit 19@386kbit|386kbit 20@256kbit|256kbit 21@256kbit|256kbit 22@256kbit|256kbit 23@256kbit|256kbit 24@256kbit|256kbit 25@256kbit|256kbit 26@256kbit|256kbit 27@256kbit|256kbit 28@256kbit|256kbit 29@256kbit|256kbit 30@256kbit|256kbit" Yeah, it''s very ugly. That''s because when I started you didn''t even have the possibility to specify ceil rates like this (the original idea was to give every flatmate an equal sized piece of the cake). USERS="1 2 3 4 5" is all you need for that... HTH Andreas