hi i''m playing with traffic shaping. trying to slowdown http uploads larger than 1mbyte - but somehow it''s not working. googling did reveal the shorewall docs about tcrules CONNBYTES but no "real life" exmples from others. my tcdevices/tcrules/tcclasses and full "shorewall show (tc|mangle)" output: http://pastebin.com/i2MTDTYq eth0 is my internet connection. using shorewall 4.4.11 on debian lenny with perl 5.10.0. I do see that the connection is marked as 4:> shorewall show connections | grep 87.238.86.135tcp 6 431999 ESTABLISHED src=192.168.236.31 \ dst=87.238.86.135 \ sport=45331 \ dport=80 packets=63029 bytes=94475296 \ src=87.238.86.135 dst=80.219.106.215 \ sport=80 dport=45331 packets=37784 \ bytes=1850659 [ASSURED] mark=4 use=1 but the excerpt from "shorewall show tc" shows no traffic on this class: class htb 1:14 parent 1:1 leaf 5: prio 4 quantum 3115 rate 162000bit \ ceil 1040Kbit burst 1620b/8 mpu 0b overhead 0b \ cburst 1729b/8 mpu 0b overhead 0b level 0 Sent 0 bytes 0 pkt (dropped 0, overlimits 0 requeues 0) rate 0bit 0pps backlog 0b 0p requeues 0 lended: 0 borrowed: 0 giants: 0 tokens: 78125 ctokens: 12995 the sip tcrule is working (Sent 24632 bytes 367 pkt ) . maybe someone has a hint why there is no traffic on class 4 or how to get the "large" uploads to move to class 4? - Thomas ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can''t live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev
Am Fri, 06 Aug 2010 06:48:48 +0000 schrieb Thomas Mueller:> hi > > i''m playing with traffic shaping. trying to slowdown http uploads larger > than 1mbyte - but somehow it''s not working. > > googling did reveal the shorewall docs about tcrules CONNBYTES but no > "real life" exmples from others. > > my tcdevices/tcrules/tcclasses and full "shorewall show (tc|mangle)" > output: http://pastebin.com/i2MTDTYq >ok, I think a can finally answer my own question. as soon as I replace 4 with 4:T in tcrules it starts working! is CONNBYTES is just working with :T ? - Thomas ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can''t live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev
On Fri, 2010-08-06 at 06:48 +0000, Thomas Mueller wrote:> hi > > i''m playing with traffic shaping. trying to slowdown http uploads larger > than 1mbyteInteresting. To diverge slightly OT for a bit, can I ask what the administrative policy is behind this? i.e. what''s the point of slowing down HTTP uploads only after 1MB of transfer? I guess I find this interesting because I''m always of the notion that the bandwidth is there to be used, not limited and as long as QOS requirements (i.e. VIOP, etc.) are being met and bandwidth is being shared equally (which it should by default) why not let the users use/have as much as they want/need? b. ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can''t live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev
On 8/6/10 4:29 AM, Thomas Mueller wrote:> Am Fri, 06 Aug 2010 06:48:48 +0000 schrieb Thomas Mueller: > >> hi >> >> i''m playing with traffic shaping. trying to slowdown http uploads larger >> than 1mbyte - but somehow it''s not working. >> >> googling did reveal the shorewall docs about tcrules CONNBYTES but no >> "real life" exmples from others. >> >> my tcdevices/tcrules/tcclasses and full "shorewall show (tc|mangle)" >> output: http://pastebin.com/i2MTDTYq >> > > ok, I think a can finally answer my own question. > > as soon as I replace 4 with 4:T in tcrules it starts working! > > is CONNBYTES is just working with :T ?You must use :T if you are testing this from the firewall itself. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can''t live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev
Am Fri, 06 Aug 2010 07:57:31 -0400 schrieb Brian J. Murrell:> On Fri, 2010-08-06 at 06:48 +0000, Thomas Mueller wrote: >> hi >> >> i''m playing with traffic shaping. trying to slowdown http uploads >> larger than 1mbyte > > Interesting. To diverge slightly OT for a bit, can I ask what the > administrative policy is behind this? i.e. what''s the point of slowing > down HTTP uploads only after 1MB of transfer? > > I guess I find this interesting because I''m always of the notion that > the bandwidth is there to be used, not limited and as long as QOS > requirements (i.e. VIOP, etc.) are being met and bandwidth is being > shared equally (which it should by default) why not let the users > use/have as much as they want/need?as I wrote - i''m just playing around right now. I made a backup of my workstation (40gb) and started uploading it to amazon s3. my internet connection was dead slow while uploading so i got the idea to slow down the upload. so no company policy here. :) but I can think of another use case: website with some larger files to download. to not disturb the "masses" just viewing plain html pages (<1mbyte) one slows down the larger downloads. IMHO as long as bandwith available with CEIL set to full they should get full speed. - Thomas ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can''t live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev
Am Fri, 06 Aug 2010 06:55:47 -0700 schrieb Tom Eastep:> On 8/6/10 4:29 AM, Thomas Mueller wrote: >> Am Fri, 06 Aug 2010 06:48:48 +0000 schrieb Thomas Mueller: >> >>> hi >>> >>> i''m playing with traffic shaping. trying to slowdown http uploads >>> larger than 1mbyte - but somehow it''s not working. >>> >>> googling did reveal the shorewall docs about tcrules CONNBYTES but no >>> "real life" exmples from others. >>> >>> my tcdevices/tcrules/tcclasses and full "shorewall show (tc|mangle)" >>> output: http://pastebin.com/i2MTDTYq >>> >>> >> ok, I think a can finally answer my own question. >> >> as soon as I replace 4 with 4:T in tcrules it starts working! >> >> is CONNBYTES is just working with :T ? > > You must use :T if you are testing this from the firewall itself. >actually the traffic doesn''t originate from FW: FW: 192.168.236.1 Uploader: 192.168.236.31 - Thomas ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can''t live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev
On Fri, 2010-08-06 at 13:58 +0000, Thomas Mueller wrote:> > as I wrote - i''m just playing around right now. I made a backup of my > workstation (40gb) and started uploading it to amazon s3. my internet > connection was dead slow while uploading so i got the idea to slow down > the upload. so no company policy here. :)Fair enough.> but I can think of another use case: website with some larger files to > download. to not disturb the "masses" just viewing plain html pages > (<1mbyte) one slows down the larger downloads. IMHO as long as bandwith > available with CEIL set to full they should get full speed.Ahh. I might be showing my lack-of-complete-understanding for the TC syntax. I agree, that as long as you are only "de-prioritizing" (i.e. letting them have full bandwidth if there are no other users, but allowing other users'' packets to take priority) packets in a connection after 1MB, that can make sense. b. ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can''t live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev
On 8/6/10 4:57 AM, Brian J. Murrell wrote:> > I guess I find this interesting because I''m always of the notion that > the bandwidth is there to be used, not limited and as long as QOS > requirements (i.e. VIOP, etc.) are being met and bandwidth is being > shared equally (which it should by default) why not let the users > use/have as much as they want/need?Brian, Your approach and Thomas''s are not incompatible. By using CONNBYTES, Thomas is able to move large uploads into a lower TC class. That doesn''t mean that such uploads will be slower if there is no congestion. But when there *is* congestion, slowing down these large uploads can improve the upload experience for other users and reduce the overall bandwidth requirement for the site. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can''t live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev
On Fri, 2010-08-06 at 07:08 -0700, Tom Eastep wrote:> > Brian,Hey Tom,> Your approach and Thomas''s are not incompatible. By using CONNBYTES, > Thomas is able to move large uploads into a lower TC class. That doesn''t > mean that such uploads will be slower if there is no congestion.Indeed. That was the bit of information I was not gleaning from his post (most likely my fault, not his). I thought he was actually imposing a "speed penalty" on these >1MB uploads rather than just de-prioritizing them. Indeed, I can see a lot of reason for de-prioritizing (but not penalizing).> But > when there *is* congestion, slowing down these large uploads can improve > the upload experience for other users and reduce the overall bandwidth > requirement for the site.Indeed. Cheers, b. ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can''t live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev
On 8/6/10 7:00 AM, Thomas Mueller wrote:> > FW: 192.168.236.1 > Uploader: 192.168.236.31 >Then the CONNBYTES rule must be in the FORWARD chain (:F) or in the POSTROUTING chain (:T). By default (with MARK_IN_FORWARD_CHAIN=No), marking is done in the PREROUTING chain. Because of limitations in Linux traffic shaping which have only recently been eliminated, Shorewall clears all marks in forwarded packets after they have been routed. Beginning with Shorewall 4.4.10, you can set CLEAR_FORWARD_MARKS=No in shorewall.conf to prevent the marks from being cleared after routing. This is only allowed if your iproute and kernel are recent enough. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can''t live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev
Am Fri, 06 Aug 2010 07:20:01 -0700 schrieb Tom Eastep:> On 8/6/10 7:00 AM, Thomas Mueller wrote: > > >> FW: 192.168.236.1 >> Uploader: 192.168.236.31 >> >> > Then the CONNBYTES rule must be in the FORWARD chain (:F) or in the > POSTROUTING chain (:T). By default (with MARK_IN_FORWARD_CHAIN=No), > marking is done in the PREROUTING chain. Because of limitations in Linux > traffic shaping which have only recently been eliminated, Shorewall > clears all marks in forwarded packets after they have been routed. > > Beginning with Shorewall 4.4.10, you can set CLEAR_FORWARD_MARKS=No in > shorewall.conf to prevent the marks from being cleared after routing. > This is only allowed if your iproute and kernel are recent enough. > > -Tomfrom man shorewall.ceph (4.4.11) FORWARD_CLEAR_MARK={Yes|No} Added in Shorewall 4.4.11 Beta 3. Traditionally, Shorewall has cleared the packet mark in the first rule in the mangle FORWARD chain. This behavior is maintained with the default setting of this option (CLEAR_FORWARD_MARK=Yes). If FORWARD_CLEAR_MARK is set to ´No´, packet marks set in the mangle PREROUTING chain are retained in the FORWARD chains. did you mean FORWARD_CLEAR_MARK ? if yes is CLEAR_FORWARD_MARK a typo in the man page? - Thomas ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can't live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
On 8/6/10 7:55 AM, Thomas Mueller wrote:> > > > did you mean FORWARD_CLEAR_MARK ? if yes is CLEAR_FORWARD_MARK a typo in > the man page? >I meant FORWARD_CLEAR_MARK. The typo is corrected in Beta4. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ This SF.net email is sponsored by Make an app they can''t live without Enter the BlackBerry Developer Challenge http://p.sf.net/sfu/RIM-dev2dev