I''ve been a long-time user of Shorewall and I want to upgrade my traffic shaping rules if possible, especially with the introduction of ifb support in newer versions. I''m currently running 4.2, using the "Wondershaper-type" configuration found at http://www.shorewall.net/traffic_shaping.htm based around my own setup. Although I have been using iptables since the ipchains days, I''m still a newbie on the usage of tc and how Shorewall uses it. This is all just for my own home setup. The problem I''m having now is that my upload can be saturated enough to cause extreme latency, and I wish to prevent that without having to rely on simply throttling back the speed. I am a residential FiOS user with a 15/2mbit plan. I have Torrent, VNC, SSH, and HTTP servers running on the network open to the outside, and use VoIP through BroadVoice. Here''s what I''m currently using, any suggestions on what to do or where to go to look for more info would be helpful: # # Shorewall version 3.4 - Tcdevices File # # For information about entries in this file, type "man shorewall-tcdevices" # # See http://shorewall.net/traffic_shaping.htm for additional information. # ############################################################################ ### #INTERFACE IN-BANDWITH OUT-BANDWIDTH REDIRECTED # INTERFACES $EXT_IF 15360kbit 2048kbit #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE # # Shorewall version 3.4 - Tcclasses File # # For information about entries in this file, type "man shorewall-tcclasses" # # See http://shorewall.net/traffic_shaping.htm for additional information. # ############################################################################ ### #INTERFACE MARK RATE CEIL PRIORITY OPTIONS # VoIP $EXT_IF 1 100kbit 180kbit 1 tos=0x68/0xfc,tos=0xb8/0xfc $EXT_IF 2 full/4 full 2 tcp-ack,tos-minimize-delay $EXT_IF 3 full/4 full 3 default $EXT_IF 4 full/8 full*8/10 4 #$EXT_IF 1 100kbit 180kbit 1 tos=0x68/0xfc,tos=0xb8/0xfc #$EXT_IF 2 full full 2 tcp-ack,tos-minimize-delay #$EXT_IF 3 9*full/10 9*full/10 3 default #$EXT_IF 4 8*full/10 8*full/10 3 #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE # Shorewall version 3.4 - Tcrules File # # For information about entries in this file, type "man shorewall-tcrules" # # See http://shorewall.net/traffic_shaping.htm for additional information. # For usage in selecting among multiple ISPs, see # http://shorewall.net/MultiISP.html # # See http://shorewall.net/PacketMarking.html for a detailed description of # the Netfilter/Shorewall packet marking mechanism. # ############################################################################ ### #MARK SOURCE DEST PROTO DEST SOURCE USER TEST LENGT$ # PORT(S) PORT(S) 2:F 0.0.0.0/0 0.0.0.0/0 icmp echo-request 2:F 0.0.0.0/0 0.0.0.0/0 icmp echo-reply 4 $INT_IF:0.0.0.0/0 $EXT_IF:0.0.0.0/0 tcp $PORT_WOWTORRENT,$PORT_HTTPSERVER,$PORT_TORRENT 4 $EXT_IF:0.0.0.0/0 $INT_IF:0.0.0.0/0 tcp - $PORT_WOWTORRENT,$PORT_HTTPSERVER,$PORT_TORRENT 4 $INT_IF:0.0.0.0/0 $EXT_IF:0.0.0.0/0 udp $PORT_TORRENT 4 $EXT_IF:0.0.0.0/0 $INT_IF:0.0.0.0/0 udp - $PORT_TORRENT #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword
Hi Newbie, When it comes to traffic shaping I am also a noob. But I have spotted one potential weakness... Read on. GeneralNMX wrote:> The problem I''m having now is that my > upload can be saturated enough to cause extreme latency, and I wish to > prevent that without having to rely on simply throttling back the speed. I > am a residential FiOS user with a 15/2mbit plan. > > #INTERFACE IN-BANDWITH OUT-BANDWIDTH REDIRECTED > $EXT_IF 15360kbit 2048kbitIn order to be able to traffic shape, you HAVE to throttle outgoing traffic. ("Outgoing" meaning "traffic leaving the firewall machine in any direction".) In other words: The IN-BANDWITH and OUT-BANDWIDTH settings must be lower than all other limitations. Otherwise packets will be buffered, leading to latency. Try setting OUT-BANDWIDTH to, say, 1500 and work your way up from there. BR /Martin ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword
I thought this was the point of tcclasses and marking packets to only use a portion of the bandwidth? I was hoping Shorewall could help me use all my bandwidth. Maybe I''m just greedy and want all the theoretically available bandwidth to myself. Alright I''ll dock it 500kbit on both ends like I did back when I was using Wondershaper (so long, long ago). -----Original Message----- From: Martin Leben [mailto:ml060223@leben.nu] Sent: Monday, January 19, 2009 10:22 PM To: shorewall-users@lists.sourceforge.net Subject: Re: [Shorewall-users] Traffic Shaping Newbie Hi Newbie, When it comes to traffic shaping I am also a noob. But I have spotted one potential weakness... Read on. GeneralNMX wrote:> The problem I''m having now is that my > upload can be saturated enough to cause extreme latency, and I wish to > prevent that without having to rely on simply throttling back the speed. I > am a residential FiOS user with a 15/2mbit plan. > > #INTERFACE IN-BANDWITH OUT-BANDWIDTH REDIRECTED > $EXT_IF 15360kbit 2048kbitIn order to be able to traffic shape, you HAVE to throttle outgoing traffic. ("Outgoing" meaning "traffic leaving the firewall machine in any direction".) In other words: The IN-BANDWITH and OUT-BANDWIDTH settings must be lower than all other limitations. Otherwise packets will be buffered, leading to latency. Try setting OUT-BANDWIDTH to, say, 1500 and work your way up from there. BR /Martin ---------------------------------------------------------------------------- -- This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword
GeneralNMX wrote:>I thought this was the point of tcclasses and marking packets to only use a >portion of the bandwidth? I was hoping Shorewall could help me use all my >bandwidth. Maybe I''m just greedy and want all the theoretically available >bandwidth to myself. Alright I''ll dock it 500kbit on both ends like I did >back when I was using Wondershaper (so long, long ago).But you MUST throttle your traffic to LESS than (or equal to) the upstream bandwidth - even if it''s only 1 byte/s slower. And every level of your prioritisation must add up to less than (or equal) the next level up. The reason shaping/prioritation is working is that you don''t allow any queue of traffic to build up that you don''t control. In your shaper, you control the queues, so you can let a queue build up for (eg) p2p traffic because you control the mechanisms that will take care of sending other stuff out ahead of it. If you send outbound traffic any faster than your uplink speed then queues can build up that you don''t control and your latency suffers on high priority traffic. The next thing you need to remember is that your effective link speed may well be very different to the sync speed of your modem/whatever. At work we have the luxury, at a cost I might add, of an uncontended and unlimited service (we do hosting) which measn we know exactly what we can shove up the wire. At home, I''m on an ADSL service where there is no guaranteed speed - the actual throughput will be lower when others are using their connections, so I have to guess at what max speeds (down and up) will mostly avoid me hitting restrictions due to contention in my ISPs backhaul. In practice, I set my up and down speed limits somewhat below the sync speed of the ADSL modem to allow for this contention. -- Simon Hobson Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed author Gladys Hobson. Novels - poetry - short stories - ideal as Christmas stocking fillers. Some available as e-books. ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword
Simon Hobson wrote:> > The next thing you need to remember is that your effective link speed > may well be very different to the sync speed of your modem/whatever. > At work we have the luxury, at a cost I might add, of an uncontended > and unlimited service (we do hosting) which measn we know exactly > what we can shove up the wire. At home, I''m on an ADSL service where > there is no guaranteed speed - the actual throughput will be lower > when others are using their connections, so I have to guess at what > max speeds (down and up) will mostly avoid me hitting restrictions > due to contention in my ISPs backhaul. In practice, I set my up and > down speed limits somewhat below the sync speed of the ADSL modem to > allow for this contention. >The description of IN-BANDWIDTH in the Shorewall TC doc includes a procedure for attempting to find an appropriate setting for that parameter. ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword
> ... usage of tc and how Shorewall uses it ...Here''s a short&sweet (but not entirely accurate) description: Traffic is divided into classes using your rules, then those classes are metered using HTB. Within each and every class an SFQ makes sure everybody gets a fair share.> ... back when I was using Wondershaper (so long, long ago)....It''s probably obvious but I''ll reiterate it anyway-- Wondershaper is great for its intended purpose: a single-user system that wants to retain snappy interactivity even when a download is in progress. But in my experience, it''s the wrong approach when firewalling a whole LAN, especially if the primary concern is web browsability (or things like VOiP and Chat and P2P) rather than straightforward downloads.> ... I want to upgrade my traffic shaping rules if possible ...As another example, what I use is documented at http://www.ckollars.org/shaping.html> ... I have been using iptables since the ipchains days ...IMHO, the one thing not to do is "mix" IPtables and Shorewall, keep it one or the other. Shorewall translates your specifications into IPtables rules (but often not in obvious ways). And it implements those specifications using the regular IPtables mechanism, so the capabilities are (almost) the same.> ... I thought this was the point of tcclasses and marking packets > to only use a portion of the bandwidth? ...Huh? Did you mean traffic shaping tries to ensure no one packet flow monopolizes the bandwidth? And it does this by as necessary restricting each packet flow to only a portion of the bandwidth?> ...use all my bandwidth ...HTB (also if driven by Shorewall) has a provision for donating "extra" bandwidth to the next lower class. So with reasonable specifications in ''tcclasses'' you can use all your bandwidth.> ... upload can be saturated enough to cause extreme latency ...With traffic shaping, it''s often hard to really grasp that it''s almost impossible to shape INcoming traffic _directly_. An _indirect_ way to prevent saturation by a download (controlling what you can) is to ration the ACKs for that operation. Slow the outgoing ACKs, indirectly slow the incoming download.> ... introduction of ifb support in newer versions ...YMMV. My experience is it''s more than possible to do everything I want wihOUT ifb. Added convenience? probably; More likely to work even with an ill-thought-out configuration? yes; Added functionality? -?- thanks! -Chuck Kollars ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword
Chuck Kollars wrote:> > As another example, what I use is documented at http://www.ckollars.org/shaping.html >Good writeup, Chuck. I''ve taken the liberty of adding a link to it from the Shorewall Traffic Shaping page. A couple of comments, though: a) It is doubtful that all of the UDP ports that you are specifying are needed (20, 21, and 110) come to immediately to mind. b) TCP port 20 is only a destination port for ACK packets. FTP servers bind to that port for active mode data connections. c) Your rules assume that no servers are running behind the Shorewall box since only requests with the listed DEST ports are being marked. Responses from local servers have the reserved ports as their SOURCE port. So, for example, outgoing responses from a web server have SOURCE port 80 (HTTP) or 443 (HTTPS). ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword
Wow! A wave of helpfulness. You''re all pretty knowledgeable and receptive.>From Chuck''s write-up, it seems I''m already doing a good portion of theright stuff. This is just a home network, not a business network, and web surfing isn''t really the highest priority. As I said in the original email, we do have some servers on this line: VoIP, HTTP, SSH, VNC, etc., while also doing heavy torrenting (legal downloads, of course). It never occurred to me to stop throttling my download in tcdevices and see how my ISP takes care of that. I now have 20mbit/5mbit after a $10/month upgrade; usually more downlink then I need. And not only do I have FiOS, but I have it hooked up via a Cat 5e cable directly to my little Debian box running Shorewall. I''m definitely already blessed in many ways, so I''m just being greedy. I''ll try to stress my network as much as I can for a month or so, logging the results to look for specific improvements. -----Original Message----- From: Shorewall Guy [mailto:shorewalljunky@comcast.net] Sent: Tuesday, January 20, 2009 2:15 PM To: ckollars9@yahoo.com; Shorewall Users Subject: Re: [Shorewall-users] Traffic Shaping Newbie Chuck Kollars wrote:> > As another example, what I use is documented athttp://www.ckollars.org/shaping.html>Good writeup, Chuck. I''ve taken the liberty of adding a link to it from the Shorewall Traffic Shaping page. A couple of comments, though: a) It is doubtful that all of the UDP ports that you are specifying are needed (20, 21, and 110) come to immediately to mind. b) TCP port 20 is only a destination port for ACK packets. FTP servers bind to that port for active mode data connections. c) Your rules assume that no servers are running behind the Shorewall box since only requests with the listed DEST ports are being marked. Responses from local servers have the reserved ports as their SOURCE port. So, for example, outgoing responses from a web server have SOURCE port 80 (HTTP) or 443 (HTTPS). ---------------------------------------------------------------------------- -- This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users ------------------------------------------------------------------------------ This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword