I''ve got many machines behind a Shorewall Firewall that among other things NATs them all. I want to add some sort of Traffic Control that will give each computer a very roughly equal slice of my Internet bandwidth. So I''ve turned on Shorewall TC. It works as expected. But there seems to be a loophole that can allow a few computers to use way way more than their fair share of bandwidth despite the TC. For example a computer that ran BitTorrent would (in my mind) abuse their capability by having their say 14 connections to different outside machines treated as 14 separate flows by the SFQ (Stochastic Fair Queueing) in the kernel and so get 14 turns (!) during every SFQ pass through its hash buckets. (Meanwhile computers browsing the web would get only one turn!) What can I do to treat each _computer_ rather than each _flow_ as a user of bandwidth? Any suggestions? thanks! (At first I thought tweaking the SFQ in the kernel was all that I needed. Shorewall TC would continue to function exactly the same without even knowing the SFQ under it was behaving differently. Fortunately for me SFQ is a loadable module that''s fairly straightforward to tweak and replace. But: all my inside computers have already undergone NAT masquerading by then, so as I understand it all the packets have the _same_ source IP address [the firewall itself], and different source ports indicate different _flows_ not different _computers_. As a result there''s not much SFQ-like code can do even with reasonable modifications.) -- Chuck Kollars http://www.ckollars.org/dragon.html ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
I''ve got many machines behind a Shorewall Firewall
that among other things NATs them all. I want to add
some sort of Traffic Control that will give each
computer a very roughly equal slice of my Internet
bandwidth. So I''ve started by turning on Shorewall TC.
It works as expected.
But, there seems to be a loophole that can allow a few
computers to use way way more than their fair share of
bandwidth despite the TC. For example a computer that
ran BitTorrent would (in my mind:-) abuse their
capability by having their say 14 connections to
different outside machines treated as 14 separate
flows by the SFQ (Stochastic Fair Queueing) in the
kernel and so get 14 turns (!) during every SFQ pass
through its hash buckets. (Meanwhile computers
browsing the web would get only one turn!)
What can I do to treat each _computer_ rather than
each _flow_ as a user of bandwidth? Any suggestions?
thanks!
(At first I thought tweaking the SFQ in the kernel was
all that I needed. Shorewall TC would continue to
function exactly the same without even knowing the SFQ
under it was behaving differently. Fortunately for me
SFQ is a loadable module that''s fairly straightforward
to tweak and replace.
But: all my inside computers have already undergone
NAT masquerading by then, so as I understand it all
the packets have the _same_ source IP address [the
firewall itself], and different source ports indicate
different _flows_ not different _computers_. As a
result, there''s not much SFQ-like code can do even
with reasonable modifications. ...Or is there?)
--
Chuck Kollars
http://www.ckollars.org/dragon.html
____________________________________________________________________________________
Building a website is a piece of cake. Yahoo! Small Business gives you all the
tools to get online.
http://smallbusiness.yahoo.com/webhosting
-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems? Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/
Chuck Kollars wrote:>I''ve got many machines behind a Shorewall Firewall >that among other things NATs them all. I want to add >some sort of Traffic Control that will give each >computer a very roughly equal slice of my Internet >bandwidth. So I''ve started by turning on Shorewall TC. >It works as expected. > >But, there seems to be a loophole that can allow a few >computers to use way way more than their fair share of >bandwidth despite the TC. For example a computer that >ran BitTorrent would (in my mind:-) abuse their >capability by having their say 14 connections to >different outside machines treated as 14 separate >flows by the SFQ (Stochastic Fair Queueing) in the >kernel and so get 14 turns (!) during every SFQ pass >through its hash buckets. (Meanwhile computers >browsing the web would get only one turn!) > >What can I do to treat each _computer_ rather than >each _flow_ as a user of bandwidth? Any suggestions?Well a first suggestion has to be to apply traffic prioritisation - see http://lartc.org/howto/lartc.cookbook.ultimate-tc.html There is an example on the Shorewall web pages to implement this with Shorewall instead of by direct manipulation of iptables. This would allow you to lower the priority of ''bulk'' traffic such as smtp and bittorrent and limit their ability to affect performance of regular activities (eg web browsing) and high priorities (like VoIP). At work we got to the stage where the VoIP became unusable during the afternoons and this made it work again. If that isn''t good enough, then the only other way I can think of would be to create a queue per (active) internal address - but that seems like a lot of work, and as you say may be too late in the chain. I do believe you should be able to mark packets in ingress though, and apply tc based on these marks. ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
On this page: http://www.shorewall.net/manpages/shorewall.html it provides information on the SHOREWALL_COMPILER (SHOREWALL_COMPILER in shorewall.conf(5).). Clicking this link takes you to: http://www.shorewall.net/manpages/shorewall.conf.html However there''s no information on the "SHOREWALL_COMPILER" setting. There is no information on this with the man pages stored locally either. I did find the information I needed by searching and found it here: http://www.shorewall.net/Shorewall-4.html Just thought I would give you a heads up. Thanks for a great firewall! I''ve been using Shorewall for years! John _________________________________________________________________ See what you’re getting into…before you go there http://newlivehotmail.com/?ocid=TXT_TAGHM_migration_HM_viral_preview_0507 --===============0259942371=Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/ --===============0259942371=Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline
J and T wrote:> On this page: > http://www.shorewall.net/manpages/shorewall.html > > it provides information on the SHOREWALL_COMPILER (SHOREWALL_COMPILER in > shorewall.conf(5).). Clicking this link takes you to: > > http://www.shorewall.net/manpages/shorewall.conf.html > > However there''s no information on the "SHOREWALL_COMPILER" setting. > > There is no information on this with the man pages stored locally > either. I did find the information I needed by searching and found it here: > > http://www.shorewall.net/Shorewall-4.html > > Just thought I would give you a heads up. >Thanks, John. In the future, please report this type of problem to the webmaster. Posts to the mailing list are sent to 950+ subscribers, only two or three of which can actually do anything about the problem.> Thanks for a great firewall! I''ve been using Shorewall for years!You''re welcome. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/
Chuck Kollars wrote:>I unfortunately didn''t explain in my post (already too >long:-) that I''m already prioritizing traffic, that >I''m concerned with the _next_ step, with the >possibility that some "bulk" traffic will shut out >other "bulk" traffic.TBH I wouldn''t worry about it - unless you have some users who downloads LOADS then the worst case is that some downloads will take longer to finish. If you are already applying prioritisation then the bulk downloads won''t affect interactive traffic.>As I understand it, Shorewall >uses HTB to enforce priorities, then uses SFQ within >each priority level to try to enfore fairness. > >As I understand it, I can (and do) use "ingress" >marking to note priorities ...but there doesn''t seem >to be any reasonable way to distinguish different >source nodes. (Except of course using a different mark >for every Source IP, which with several hundred inside >systems is far too weird and difficult.)You understand correctly.>It seems to me that if my SFQ modifications could do a >quick lookup of the original Source IP in the CONNMARK >table, I could get the behavior I want. But otherwise, >as you confirm if I understand you correctly, there >really isn''t any way. All I can do is hope the >prioritization is sufficient and not worry about also >trying to solve the follow-on problem.Well you could look into the code and see if there might be a way to modify the SFQ code ... ------------------------------------------------------------------------- This SF.net email is sponsored by: Splunk Inc. Still grepping through log files to find problems? Stop. Now Search log events and configuration files using AJAX and a browser. Download your FREE copy of Splunk now >> http://get.splunk.com/