Jason Tackaberry
2002-Oct-10 01:54 UTC
Can''t keep up with all these file sharing programs!
Hi everyone, I''m using HTB to shape traffic for students in our residences. We''re an extremely small college (about 50 Internet users in our residences) and we don''t have a good deal of bandwidth to work with, so I must do what I can to make what we do have tolerable to our students. I am right now using the following approach: I have allotted a portion of our total bandwidth (R) to the residence subnet on the upstream interface on our router. This class is sub-divided into two classes: a p2p class for all those pesky file sharing programs, which has a ceiling of about R/2, and an "everything else" class, which has a guaranteed rate of R/2, and a ceililng of R. I have put SYN and ACK packets in a separate class (under root) to improve responsiveness. In theory, this scheme works pretty good. The problem is that every day some of these p2p programs are using different ports, and they manage to suck up all available downstream bandwidth. So, the student who wants to send their friend a file over ICQ is going to get starved by every other student running Kazaa-du-jour. Now it would be _really_ nice if there was some ability to examine packets at layer 7 to determine what class a particular session belongs in (like, for instance, the way Packeteer''s Packet Shaper works). I''m assuming I can''t get this functionality (unless I write it myself), so can someone suggest a remedy to my problem? Is there some magic adjustment I can make? Or, perhaps I should try a different approach, and give each IP a guaranteed rate? The only drawback I see with this is that with 50 users, I could only guarantee each user 5kbps. :) Any guidance would be appreciated. Best, Jason. -- Jason Tackaberry :: tack@auc.ca :: 705-949-2301 x330 Academic Computing Support Specialist Information Technology Services Algoma University College :: www.auc.ca _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Kirby C. Bohling
2002-Oct-10 04:00 UTC
Re: Can''t keep up with all these file sharing programs!
On Wed, 2002-10-09 at 20:54, Jason Tackaberry wrote:> Hi everyone, > > I''m using HTB to shape traffic for students in our residences. We''re an > extremely small college (about 50 Internet users in our residences) and > we don''t have a good deal of bandwidth to work with, so I must do what I > can to make what we do have tolerable to our students. > > I am right now using the following approach: I have allotted a portion > of our total bandwidth (R) to the residence subnet on the upstream > interface on our router. This class is sub-divided into two classes: a > p2p class for all those pesky file sharing programs, which has a ceiling > of about R/2, and an "everything else" class, which has a guaranteed > rate of R/2, and a ceililng of R. I have put SYN and ACK packets in a > separate class (under root) to improve responsiveness. > > In theory, this scheme works pretty good. The problem is that every day > some of these p2p programs are using different ports, and they manage to > suck up all available downstream bandwidth. So, the student who wants > to send their friend a file over ICQ is going to get starved by every > other student running Kazaa-du-jour. > > Now it would be _really_ nice if there was some ability to examine > packets at layer 7 to determine what class a particular session belongs > in (like, for instance, the way Packeteer''s Packet Shaper works). I''m > assuming I can''t get this functionality (unless I write it myself), so > can someone suggest a remedy to my problem? Is there some magic > adjustment I can make? Or, perhaps I should try a different approach, > and give each IP a guaranteed rate? The only drawback I see with this > is that with 50 users, I could only guarantee each user 5kbps. :) > > Any guidance would be appreciated. >Jason, It sounds like your using a black list to get you moved into the "P2P" bandwidth.... Why not turn it around and review /etc/services and say: http,smtp,pop3,imap,news,https,rsync,icq,irc,ssh and other services get into the white list of "Non-P2P" bandwidth. It''s much easier to qualify the things you like, instead of the things you don''t like in this case. I''d also shift the percentage to 75/25 whitelist/blacklist. Now if anybody abuses this, by trying to run P2P over a well-known port, there are ways of solving that too. First just bandwidth limit them into you get 2.5K unless there is bandwidth to spare.... Another idea, is keep stats on "abused ports". Any source or dest port that consumes more then X% of the total bandwidth gets put onto the blacklist ports. Script the whole stats, and adding removing ports from the list. Possibly just keep port 80 on the it''s okay post list it will always exceed X% for reasonable values of X. There''s also a certain amout of fairness in just splitting the bandwidth guaranteed evenly. It''s about as fair as you can be. An interesting thing could be split the bandwith cap inversely proportional to usage. So for someone who rarely uses the network connection, they have a higher hard cap then someone who uses it constantly. Just order everyones network usage, give the first 5% guaranteed access to 30% of the bandwidth. Give the next 60% guaranteed access to 60%, give the last 20% access to 10% of the bandwidth as a guaranteed limit. So when everyone is on, the people who generally don''t use it much get on an abundance of bandwidth during the few times they want it. Maybe do that with 80% of the bandwidth, leave the other 20% uncontrolled for anyone to use at any rate they can get it at just so there is burstable bandwidth available to anybody when they need it. The people who use more then their fair share most of the day, can wait while the people who generally don''t take their portion are attempting to use it. This is even more fair then picking "bad" services. It doesn''t make any difference if I''m consuming all the bandwidth downloading web pages to build the next google index, or if I''m consuming all the bandwidth using Kazaa. I''m still using too much of the shared resource. What I''m using it on is relatively irrelavent. Me using it for anything keeps someone else from using it productively. Keep the stats and re-categorize the users say every 6/12/24 hours. Maybe do it just once a week. You might have to shuffle the percentages, and user count a bit, but it should work to give people a fair chance at using up their portion of the bandwidth when they need it. Yeah it''s a fair amount of scripting, but should be relatively stable setup once you are done with it. Thanks, Kirby> Best, > Jason. > > -- > Jason Tackaberry :: tack@auc.ca :: 705-949-2301 x330 > Academic Computing Support Specialist > Information Technology Services > Algoma University College :: www.auc.ca > > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ >-- Real Programmers view electronic multimedia files with a hex editor. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
You could try a completely different approach: First, set up an iptables rule that redirects anything outbound that''s port 80 or 20 or 21 to, say, squid or some other proxy server. Then block **everything** else going out. So for the outbound web stuff, get one of the commercial filtering packages and put that on top of the proxy server redirected above. That will do the layer 7 filtering within the context of outbound web access and will block inappropriate sites. This will also solve your Kazaah du jor problem cuz nothing will go out except legit web and ftp stuff. Just a thought - facing similar issues myself. I don''t think you can fight this problem with low level traffic shaping. - Greg Scott -----Original Message----- From: Jason Tackaberry [mailto:tack@auc.ca] Sent: Wednesday, October 09, 2002 8:55 PM To: lartc@mailman.ds9a.nl Subject: [LARTC] Can''t keep up with all these file sharing programs! Hi everyone, I''m using HTB to shape traffic for students in our residences. We''re an extremely small college (about 50 Internet users in our residences) and we don''t have a good deal of bandwidth to work with, so I must do what I can to make what we do have tolerable to our students. I am right now using the following approach: I have allotted a portion of our total bandwidth (R) to the residence subnet on the upstream interface on our router. This class is sub-divided into two classes: a p2p class for all those pesky file sharing programs, which has a ceiling of about R/2, and an "everything else" class, which has a guaranteed rate of R/2, and a ceililng of R. I have put SYN and ACK packets in a separate class (under root) to improve responsiveness. In theory, this scheme works pretty good. The problem is that every day some of these p2p programs are using different ports, and they manage to suck up all available downstream bandwidth. So, the student who wants to send their friend a file over ICQ is going to get starved by every other student running Kazaa-du-jour. Now it would be _really_ nice if there was some ability to examine packets at layer 7 to determine what class a particular session belongs in (like, for instance, the way Packeteer''s Packet Shaper works). I''m assuming I can''t get this functionality (unless I write it myself), so can someone suggest a remedy to my problem? Is there some magic adjustment I can make? Or, perhaps I should try a different approach, and give each IP a guaranteed rate? The only drawback I see with this is that with 50 users, I could only guarantee each user 5kbps. :) Any guidance would be appreciated. Best, Jason. -- Jason Tackaberry :: tack@auc.ca :: 705-949-2301 x330 Academic Computing Support Specialist Information Technology Services Algoma University College :: www.auc.ca _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Werner Almesberger
2002-Oct-10 05:00 UTC
Re: Can''t keep up with all these file sharing programs!
Jason Tackaberry wrote:> In theory, this scheme works pretty good. The problem is that every day > some of these p2p programs are using different ports,So why don''t you just turn the classification around and put everything you know to be "good" in the guaranteed class and all the rest in the junk class ?> Now it would be _really_ nice if there was some ability to examine > packets at layer 7 to determine what class a particular session belongs > in (like, for instance, the way Packeteer''s Packet Shaper works). I''m > assuming I can''t get this functionality (unless I write it myself),I suppose something along the lines of a transparent proxy would work, but I don''t know where you''d find a program that implements the kind of classification you need.> adjustment I can make? Or, perhaps I should try a different approach, > and give each IP a guaranteed rate? The only drawback I see with this > is that with 50 users, I could only guarantee each user 5kbps. :)You could give them equal (RR) shares per IP, with CBQ or HTB. That won''t help against distributed downloads (e.g. one downloader using several people''s PCs), but it should buy you some time for implementing the L7 solution :-) - Werner -- _________________________________________________________________________ / Werner Almesberger, Buenos Aires, Argentina wa@almesberger.net / /_http://www.almesberger.net/____________________________________________/ _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Michael T. Babcock
2002-Oct-10 13:50 UTC
Re: Can''t keep up with all these file sharing programs!
Jason Tackaberry wrote:>I''m using HTB to shape traffic for students in our residences. We''re an >extremely small college (about 50 Internet users in our residences) and >we don''t have a good deal of bandwidth to work with, so I must do what I >can to make what we do have tolerable to our students. > >Shape all the traffic you recognize, then leave the remaining traffic as "other" and give it whatever''s left. That way you don''t have to know what the other traffic is at all and you can always add more ''known'' protocols to your shaping rules as you become aware of them ... "what do you mean you''re using WAIS?" -- Michael T. Babcock C.T.O., FibreSpeed Ltd. http://www.fibrespeed.net/~mbabcock _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
I believe Hogwash does some kind of packet inspection and replaces data in packets. Maybe you could take alook at it. It is used for IDS but uses the technique you have wanted. Maybe it can be used for this. Mohan -----Original Message----- From: lartc-admin@mailman.ds9a.nl [mailto:lartc-admin@mailman.ds9a.nl] On Behalf Of Jason Tackaberry Sent: Thursday, October 10, 2002 7:25 AM To: lartc@mailman.ds9a.nl Subject: [LARTC] Can''t keep up with all these file sharing programs! Hi everyone, I''m using HTB to shape traffic for students in our residences. We''re an extremely small college (about 50 Internet users in our residences) and we don''t have a good deal of bandwidth to work with, so I must do what I can to make what we do have tolerable to our students. I am right now using the following approach: I have allotted a portion of our total bandwidth (R) to the residence subnet on the upstream interface on our router. This class is sub-divided into two classes: a p2p class for all those pesky file sharing programs, which has a ceiling of about R/2, and an "everything else" class, which has a guaranteed rate of R/2, and a ceililng of R. I have put SYN and ACK packets in a separate class (under root) to improve responsiveness. In theory, this scheme works pretty good. The problem is that every day some of these p2p programs are using different ports, and they manage to suck up all available downstream bandwidth. So, the student who wants to send their friend a file over ICQ is going to get starved by every other student running Kazaa-du-jour. Now it would be _really_ nice if there was some ability to examine packets at layer 7 to determine what class a particular session belongs in (like, for instance, the way Packeteer''s Packet Shaper works). I''m assuming I can''t get this functionality (unless I write it myself), so can someone suggest a remedy to my problem? Is there some magic adjustment I can make? Or, perhaps I should try a different approach, and give each IP a guaranteed rate? The only drawback I see with this is that with 50 users, I could only guarantee each user 5kbps. :) Any guidance would be appreciated. Best, Jason. -- Jason Tackaberry :: tack@auc.ca :: 705-949-2301 x330 Academic Computing Support Specialist Information Technology Services Algoma University College :: www.auc.ca _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/