Hello, I am having problems getting traffic shaping working. Everything gets entered in correctly by shorewall - that is it starts up and there are no error messages logged. The problem is when I actually test the bandwidth, it exceeds what I specify as the ceiling in tcdevices - generally by 4 times. So just for testing, I lowered things a bit - here is the only line in tcdevices:> eth0 500kbit 500kbitHere is tcclasses:> eth0 1 500kbit 500kbit > 1 tcp-ack,tos-minimize-delay > eth0 3 500kbit 500kbit 3 > eth0 4 500kbit 500kbit > 4 default > eth0 6 500kbit 500kbit 6 > eth0 7 500kbit 500kbit 7 > eth0 8 100kbit 100kbit 8Here is tcrules:> 1 0.0.0.0/0 > 206.83.209.0/25 tcp 22 > 1 206.83.209.0/25 > 0.0.0.0/0 tcp 22 > 1 0.0.0.0/0 > 0.0.0.0/0 icmp echo-request > 1 0.0.0.0/0 > 0.0.0.0/0 icmp echo-reply > 5 0.0.0.0/0 > 0.0.0.0/0 tcp 80 > 5 0.0.0.0/0 > 0.0.0.0/0 tcp 443So clearly, if I connect to this server and download a file over sftp (outgoing bandwidth), the download rate should not exceed 500kbit - at least that is my understanding. When I download, it comes in 260KB (approximately 2140kbit) - a little over 4 times the specified ceiling. I am the only one connected and downloading anything. I have tried this out in 3 different kernels - stock CentOS 4.3, latest stable 2.6.16.13 and a slightly older 2.6.15.7. I even built a new tc from latest iproute2, just in case there were problems there. Nothing seems to fix the problem. Now I am doing one strange thing in /etc/shorewall/start, I have these rules to classify bulk traffic into the three lowest priority classes in tcclasses:> /usr/local/sbin/iptables -t mangle -A tcpre -p all -m connbytes > --connbytes 500000:2000000 --connbytes-dir both --connbytes-mode bytes > -j MARK --set-mark 0x6 > /usr/local/sbin/iptables -t mangle -A tcpre -p all -m connbytes > --connbytes 2000001:25000000 --connbytes-dir both --connbytes-mode > bytes -j MARK --set-mark 0x7 > /usr/local/sbin/iptables -t mangle -A tcpre -p all -m connbytes > --connbytes 25000001: --connbytes-dir both --connbytes-mode bytes -j > MARK --set-mark 0x8But currently I have them disabled/commented to eliminate any affect they may have on my download test detailed above. I do have them enabled in my dump file: http://caffeine.contactdesigns.com/~jolt/shorewalldump.txt and you can see the rules are getting hits - despite the hits, the transfer rate did not change. So I am assuming the "--set-mark 0x8" fired, but it had no effect. QOS sort of works - that is, if I remove all traffic shaping , it transfers VERY fast (100mbit connection) - so it sort of works, just not well. Any advice is greatly appreciated because I have spent days now trying to get this working and am very frustrated. I must be missing something, but I don''t know what. Thanks, Michael P.S. My apologies if this message goes to list twice - first time I used wrong return address (wrong thunderbird identity) ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
Michael McCallister wrote:> QOS sort of works - that is, if I remove all traffic shaping , it > transfers VERY fast (100mbit connection) - so it sort of works, just not > well. Any advice is greatly appreciated because I have spent days now > trying to get this working and am very frustrated. I must be missing > something, but I don''t know what. >I have two suggestions: a) Review the thread in the Archives entitled "Traffic Shaping not Effective" which began on 3/24 of this year. IIRC, your traffic shaping experience is similar to the OP in that thread and is based on a similar mis-understanding of how it works. b) Packet marking isn''t stateful. That means that you have to mark every single output packet that you want shaped. You are testing using SFTP where the server is on the system where Shorewall runs. That means that the outbound packets have SOURCE port 22, not destination port 22. So your rules are matching only request packets, not reply packets. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
Tom Eastep wrote:> Michael McCallister wrote: > > >> QOS sort of works - that is, if I remove all traffic shaping , it >> transfers VERY fast (100mbit connection) - so it sort of works, just not >> well. Any advice is greatly appreciated because I have spent days now >> trying to get this working and am very frustrated. I must be missing >> something, but I don''t know what. >> >> > > I have two suggestions: > > a) Review the thread in the Archives entitled "Traffic Shaping not > Effective" which began on 3/24 of this year. IIRC, your traffic shaping > experience is similar to the OP in that thread and is based on a similar > mis-understanding of how it works. > > b) Packet marking isn''t stateful. That means that you have to mark every > single output packet that you want shaped. You are testing using SFTP > where the server is on the system where Shorewall runs. That means that > the outbound packets have SOURCE port 22, not destination port 22. So > your rules are matching only request packets, not reply packets. > > -Tom >Thanks Tom, I really appreciate your suggestions. I read the whole thread you mentioned. I either missed something or did not understand where my misunderstanding lies. In an effort to simplify things, I now have only the following: tcdevices: eth0 1000kbit 1000kbit tcclasses: eth0 4 1000kbit 1000kbit 4 default tcrules: 4 0.0.0.0/0 0.0.0.0/0 all So basically, I am trying to limit everything to 1000kbit just as a starting point. To review, this firewall protects servers. I am connecting to the firewall directly from an external server for my tests (on totally different network). When I download from the firewall (SSH is open on the firewall), it downloads at a little over 4000kbit. This morning I tried downloading from a server behind the firewall and it did cap it at 1000kbit. This works for me! It is just a little confusing because it sort of caps bandwidth directly from the firewall at around 4 times greater than the specified rate. This lead me to believe there was a problem and I foolishly got caught up focusing on it instead of moving on to test the speed from servers behind the firewall. Although I will not be serving anything from the firewall (its sole purpose is to handle packets), some people with a small budget might do this (i.e. home office firewall with web server). So they may end up somewhat confused like me if they encounter this behavior. Thanks for the advice on packet marking - that make sense. If there a workaround, to classifying SSH packets without u32 support? Michael ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
Michael McCallister wrote:> Tom Eastep wrote: >> Michael McCallister wrote: >> >> >>> QOS sort of works - that is, if I remove all traffic shaping , it >>> transfers VERY fast (100mbit connection) - so it sort of works, just not >>> well. Any advice is greatly appreciated because I have spent days now >>> trying to get this working and am very frustrated. I must be missing >>> something, but I don''t know what. >>> >>> >> >> I have two suggestions: >> >> a) Review the thread in the Archives entitled "Traffic Shaping not >> Effective" which began on 3/24 of this year. IIRC, your traffic shaping >> experience is similar to the OP in that thread and is based on a similar >> mis-understanding of how it works. >> >> b) Packet marking isn''t stateful. That means that you have to mark every >> single output packet that you want shaped. You are testing using SFTP >> where the server is on the system where Shorewall runs. That means that >> the outbound packets have SOURCE port 22, not destination port 22. So >> your rules are matching only request packets, not reply packets. >> >> -Tom >> > Thanks Tom, > > I really appreciate your suggestions. I read the whole thread you > mentioned. I either missed something or did not understand where my > misunderstanding lies. In an effort to simplify things, I now have only > the following: > > tcdevices: > eth0 1000kbit 1000kbit > > tcclasses: > eth0 4 1000kbit 1000kbit 4 > default >Ah -- this explains why you are unable to restrict output from a server running on the firewall.> tcrules: > 4 0.0.0.0/0 0.0.0.0/0 allThat rule marks no any packets originating on the firewall itself. NONE!!! If you want to mark packets originating on the firewall itself, you must specify $FW in the SOURCE column. From http://www.shorewall.net/traffic_shaping.html (and repeated in the comments in /etc/shorewall/tcrules): "For example, all packets for connections masqueraded to eth0 from other interfaces can be matched in a single rule with several alternative SOURCE criteria. However, a connection whose packets gets to eth0 in a different way, e.g., direct from the firewall itself, needs a different rule. Accordingly, use $FW in its own separate rule for packets originating on the firewall. In such a rule, the MARK column may NOT specify either ":P" or ":F" because marking for firewall-originated packets always occurs in the OUTPUT chain." -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
Tom Eastep wrote:> Michael McCallister wrote: > >> Tom Eastep wrote: >> >>> Michael McCallister wrote: >>> >>> >>> >>>> QOS sort of works - that is, if I remove all traffic shaping , it >>>> transfers VERY fast (100mbit connection) - so it sort of works, just not >>>> well. Any advice is greatly appreciated because I have spent days now >>>> trying to get this working and am very frustrated. I must be missing >>>> something, but I don''t know what. >>>> >>>> >>>> >>> I have two suggestions: >>> >>> a) Review the thread in the Archives entitled "Traffic Shaping not >>> Effective" which began on 3/24 of this year. IIRC, your traffic shaping >>> experience is similar to the OP in that thread and is based on a similar >>> mis-understanding of how it works. >>> >>> b) Packet marking isn''t stateful. That means that you have to mark every >>> single output packet that you want shaped. You are testing using SFTP >>> where the server is on the system where Shorewall runs. That means that >>> the outbound packets have SOURCE port 22, not destination port 22. So >>> your rules are matching only request packets, not reply packets. >>> >>> -Tom >>> >>> >> Thanks Tom, >> >> I really appreciate your suggestions. I read the whole thread you >> mentioned. I either missed something or did not understand where my >> misunderstanding lies. In an effort to simplify things, I now have only >> the following: >> >> tcdevices: >> eth0 1000kbit 1000kbit >> >> tcclasses: >> eth0 4 1000kbit 1000kbit 4 >> default >> >> > > Ah -- this explains why you are unable to restrict output from a server > running on the firewall. > > >> tcrules: >> 4 0.0.0.0/0 0.0.0.0/0 all >> > > That rule marks no any packets originating on the firewall itself. > NONE!!! If you want to mark packets originating on the firewall itself, > you must specify $FW in the SOURCE column. > > From http://www.shorewall.net/traffic_shaping.html (and repeated in the > comments in /etc/shorewall/tcrules): > > "For example, all packets for connections masqueraded to eth0 from other > interfaces can be matched in a single rule with several alternative > SOURCE criteria. However, a connection whose packets gets to eth0 in a > different way, e.g., direct from the firewall itself, needs a different > rule. > > Accordingly, use $FW in its own separate rule for packets originating on > the firewall. In such a rule, the MARK column may NOT specify either > ":P" or ":F" because marking for firewall-originated packets always > occurs in the OUTPUT chain." > > -Tom >Hi Tom, Thanks again for getting back to me. I just added the 0.0.0.0/0 rule to do a quick test and to clear out all my other rules to eliminate any affect they might have so as not to detract from my observations. I was assuming that whatever I put in tcdevices (1000kbit) would apply to the overall throughput regardless of what ended up in tcrules. If this were the case, I should not have been able to transfer at over 4000kbit when tcdevices specified 1000kbit regardless of tcrules - or so I thought. So I tried adding $FW as follows to follow up on your advice: 6 $FW 0.0.0.0/0 all Where mark 6 specifies a class/mark with100kbit - sure enough it is now transferring at 400kbit. This means that it did mark the packets correctly, but the throughput is clearly off (times four again approx.). So per my original test, this means that the figures specified in tcdevices probably do apply to all traffic (as far as throughput) regardless of tcrules - there is just some issue with traffic with source $FW (for me it is approx. times 4 whatever is specified). I have tried this in 3 different kernels - all with the same results. Anyways, this doesn''t really affect me since I am not using the firewall as a server - just thought I would share in case someone else ran into this problem too - to help them avoid becoming temporarily insane as I did trying to figure it out :-) Thanks again for your feedback, Michael ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
Michael McCallister wrote:> Tom Eastep wrote: >> Michael McCallister wrote: >> >>> Tom Eastep wrote: >>> >>>> Michael McCallister wrote: >>>> >>>> >>>> >>>>> QOS sort of works - that is, if I remove all traffic shaping , it >>>>> transfers VERY fast (100mbit connection) - so it sort of works, >>>>> just not >>>>> well. Any advice is greatly appreciated because I have spent days now >>>>> trying to get this working and am very frustrated. I must be missing >>>>> something, but I don''t know what. >>>>> >>>>> >>>> I have two suggestions: >>>> >>>> a) Review the thread in the Archives entitled "Traffic Shaping not >>>> Effective" which began on 3/24 of this year. IIRC, your traffic shaping >>>> experience is similar to the OP in that thread and is based on a >>>> similar >>>> mis-understanding of how it works. >>>> >>>> b) Packet marking isn''t stateful. That means that you have to mark >>>> every >>>> single output packet that you want shaped. You are testing using SFTP >>>> where the server is on the system where Shorewall runs. That means that >>>> the outbound packets have SOURCE port 22, not destination port 22. So >>>> your rules are matching only request packets, not reply packets. >>>> >>>> -Tom >>>> >>> Thanks Tom, >>> >>> I really appreciate your suggestions. I read the whole thread you >>> mentioned. I either missed something or did not understand where my >>> misunderstanding lies. In an effort to simplify things, I now have only >>> the following: >>> >>> tcdevices: >>> eth0 1000kbit 1000kbit >>> >>> tcclasses: >>> eth0 4 1000kbit 1000kbit >>> 4 default >>> >>> >> >> Ah -- this explains why you are unable to restrict output from a server >> running on the firewall. >> >> >>> tcrules: >>> 4 0.0.0.0/0 >>> 0.0.0.0/0 all >>> >> >> That rule marks no any packets originating on the firewall itself. >> NONE!!! If you want to mark packets originating on the firewall itself, >> you must specify $FW in the SOURCE column. >> >> From http://www.shorewall.net/traffic_shaping.html (and repeated in the >> comments in /etc/shorewall/tcrules): >> >> "For example, all packets for connections masqueraded to eth0 from other >> interfaces can be matched in a single rule with several alternative >> SOURCE criteria. However, a connection whose packets gets to eth0 in a >> different way, e.g., direct from the firewall itself, needs a different >> rule. >> >> Accordingly, use $FW in its own separate rule for packets originating on >> the firewall. In such a rule, the MARK column may NOT specify either >> ":P" or ":F" because marking for firewall-originated packets always >> occurs in the OUTPUT chain." >> >> -Tom >> > Hi Tom, > > Thanks again for getting back to me. I just added the 0.0.0.0/0 rule to > do a quick test and to clear out all my other rules to eliminate any > affect they might have so as not to detract from my observations. I was > assuming that whatever I put in tcdevices (1000kbit) would apply to the > overall throughput regardless of what ended up in tcrules. If this were > the case, I should not have been able to transfer at over 4000kbit when > tcdevices specified 1000kbit regardless of tcrules - or so I thought. > So I tried adding $FW as follows to follow up on your advice: > > 6 $FW 0.0.0.0/0 all > > Where mark 6 specifies a class/mark with100kbit - sure enough it is now > transferring at 400kbit. This means that it did mark the packets > correctly, but the throughput is clearly off (times four again > approx.). So per my original test, this means that the figures > specified in tcdevices probably do apply to all traffic (as far as > throughput) regardless of tcrules - there is just some issue with > traffic with source $FW (for me it is approx. times 4 whatever is > specified). I have tried this in 3 different kernels - all with the > same results. > > Anyways, this doesn''t really affect me since I am not using the firewall > as a server - just thought I would share in case someone else ran into > this problem too - to help them avoid becoming temporarily insane as I > did trying to figure it out :-) >Well, I just tested this and it works for me. /etc/shorewall/tcdevices (I just took the rated speed of the Fast Ethernet LAN which I didn''t measure). #INTERFACE IN-BANDWITH OUT-BANDWIDTH eth0 100000kbit 100000kbit #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE /etc/shorewall/tcclasses: #INTERFAC MARK RATE CEIL PRIORITY OPTIONS eth0 1 full full 1 tcp-ack,tos-minimize-delay eth0 2 9*full/10 9*full/10 2 default eth0 3 2*full/10 2*full/10 3 #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE /etc/shorewall/tcrules: #MARK SOURCE DEST PROTO PORT(S) CLIENT USER # PORT(S) 3 $FW 192.168.1.5 tcp - 22 #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE Fetching /home/teastep/Download/SuSE10.1/SUSE-Linux-10.1-RC3-i386-CD1.iso to SUSE-Linux-10.1-RC3-i386-CD1.iso /home/teastep/Download/SuSE10.1/SUSE-Linux-10.1-RC3-i386-CD1.iso 100% 657MB 2.2MB/s 04:59 sftp> 2.2MB/s = 2.2*8 = 17.6mbit with is approximately what I specified (20mbit). Without traffic shaping, the file transfers at 9.0MB/s I have also attached a copy of "shorewall show tc" taken while the transfer was in progress. The rate calculated by HTB at that instant was 19410Kbit. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
Tom Eastep wrote:> > I have also attached a copy of "shorewall show tc" taken while the > transfer was in progress. The rate calculated by HTB at that instant was > 19410Kbit. >That''s class 1:13 of course. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
Tom Eastep wrote:> Michael McCallister wrote: > >> Tom Eastep wrote: >> >>> Michael McCallister wrote: >>> >>> >>>> Tom Eastep wrote: >>>> >>>> >>>>> Michael McCallister wrote: >>>>> >>>>> >>>>> >>>>> >>>>>> QOS sort of works - that is, if I remove all traffic shaping , it >>>>>> transfers VERY fast (100mbit connection) - so it sort of works, >>>>>> just not >>>>>> well. Any advice is greatly appreciated because I have spent days now >>>>>> trying to get this working and am very frustrated. I must be missing >>>>>> something, but I don''t know what. >>>>>> >>>>>> >>>>>> >>>>> I have two suggestions: >>>>> >>>>> a) Review the thread in the Archives entitled "Traffic Shaping not >>>>> Effective" which began on 3/24 of this year. IIRC, your traffic shaping >>>>> experience is similar to the OP in that thread and is based on a >>>>> similar >>>>> mis-understanding of how it works. >>>>> >>>>> b) Packet marking isn''t stateful. That means that you have to mark >>>>> every >>>>> single output packet that you want shaped. You are testing using SFTP >>>>> where the server is on the system where Shorewall runs. That means that >>>>> the outbound packets have SOURCE port 22, not destination port 22. So >>>>> your rules are matching only request packets, not reply packets. >>>>> >>>>> -Tom >>>>> >>>>> >>>> Thanks Tom, >>>> >>>> I really appreciate your suggestions. I read the whole thread you >>>> mentioned. I either missed something or did not understand where my >>>> misunderstanding lies. In an effort to simplify things, I now have only >>>> the following: >>>> >>>> tcdevices: >>>> eth0 1000kbit 1000kbit >>>> >>>> tcclasses: >>>> eth0 4 1000kbit 1000kbit >>>> 4 default >>>> >>>> >>>> >>> Ah -- this explains why you are unable to restrict output from a server >>> running on the firewall. >>> >>> >>> >>>> tcrules: >>>> 4 0.0.0.0/0 >>>> 0.0.0.0/0 all >>>> >>>> >>> That rule marks no any packets originating on the firewall itself. >>> NONE!!! If you want to mark packets originating on the firewall itself, >>> you must specify $FW in the SOURCE column. >>> >>> From http://www.shorewall.net/traffic_shaping.html (and repeated in the >>> comments in /etc/shorewall/tcrules): >>> >>> "For example, all packets for connections masqueraded to eth0 from other >>> interfaces can be matched in a single rule with several alternative >>> SOURCE criteria. However, a connection whose packets gets to eth0 in a >>> different way, e.g., direct from the firewall itself, needs a different >>> rule. >>> >>> Accordingly, use $FW in its own separate rule for packets originating on >>> the firewall. In such a rule, the MARK column may NOT specify either >>> ":P" or ":F" because marking for firewall-originated packets always >>> occurs in the OUTPUT chain." >>> >>> -Tom >>> >>> >> Hi Tom, >> >> Thanks again for getting back to me. I just added the 0.0.0.0/0 rule to >> do a quick test and to clear out all my other rules to eliminate any >> affect they might have so as not to detract from my observations. I was >> assuming that whatever I put in tcdevices (1000kbit) would apply to the >> overall throughput regardless of what ended up in tcrules. If this were >> the case, I should not have been able to transfer at over 4000kbit when >> tcdevices specified 1000kbit regardless of tcrules - or so I thought. >> So I tried adding $FW as follows to follow up on your advice: >> >> 6 $FW 0.0.0.0/0 all >> >> Where mark 6 specifies a class/mark with100kbit - sure enough it is now >> transferring at 400kbit. This means that it did mark the packets >> correctly, but the throughput is clearly off (times four again >> approx.). So per my original test, this means that the figures >> specified in tcdevices probably do apply to all traffic (as far as >> throughput) regardless of tcrules - there is just some issue with >> traffic with source $FW (for me it is approx. times 4 whatever is >> specified). I have tried this in 3 different kernels - all with the >> same results. >> >> Anyways, this doesn''t really affect me since I am not using the firewall >> as a server - just thought I would share in case someone else ran into >> this problem too - to help them avoid becoming temporarily insane as I >> did trying to figure it out :-) >> >> > > Well, I just tested this and it works for me. > > /etc/shorewall/tcdevices (I just took the rated speed of the Fast > Ethernet LAN which I didn''t measure). > > #INTERFACE IN-BANDWITH OUT-BANDWIDTH > eth0 100000kbit 100000kbit > #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE > > /etc/shorewall/tcclasses: > > #INTERFAC MARK RATE CEIL PRIORITY OPTIONS > eth0 1 full full 1 tcp-ack,tos-minimize-delay > eth0 2 9*full/10 9*full/10 2 default > eth0 3 2*full/10 2*full/10 3 > #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE > > /etc/shorewall/tcrules: > > #MARK SOURCE DEST PROTO PORT(S) CLIENT USER > # PORT(S) > 3 $FW 192.168.1.5 tcp - 22 > #LAST LINE -- ADD YOUR ENTRIES BEFORE THIS ONE -- DO NOT REMOVE > > > Fetching > /home/teastep/Download/SuSE10.1/SUSE-Linux-10.1-RC3-i386-CD1.iso to > SUSE-Linux-10.1-RC3-i386-CD1.iso > /home/teastep/Download/SuSE10.1/SUSE-Linux-10.1-RC3-i386-CD1.iso > 100% 657MB > 2.2MB/s 04:59 > sftp> > > 2.2MB/s = 2.2*8 = 17.6mbit with is approximately what I specified > (20mbit). Without traffic shaping, the file transfers at 9.0MB/s > > I have also attached a copy of "shorewall show tc" taken while the > transfer was in progress. The rate calculated by HTB at that instant was > 19410Kbit. > > -Tom >Hi Tom, In that case, it must be something specific to my setup. I tried different kernels (some custom - some stock) and different iptables to try and eliminate anything that was unique to my implementation but something over here is obviously the culprit. My apologies for the distraction. Michael ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
Hi Michael
Looking at your dump, it seems that eth0 and eth1 have the same IP-address:
2: eth0: <BROADCAST,MULTICAST,UP> mtu 9000 qdisc htb qlen 1000
link/ether 00:04:23:c3:98:d0 brd ff:ff:ff:ff:ff:ff
inet 206.83.209.5/25 brd 206.83.209.127 scope global eth0
inet6 fe80::204:23ff:fec3:98d0/64 scope link
valid_lft forever preferred_lft forever
3: eth1: <BROADCAST,MULTICAST,UP> mtu 9000 qdisc pfifo_fast qlen 1000
link/ether 00:04:23:c3:98:d1 brd ff:ff:ff:ff:ff:ff
inet 206.83.209.5/32 brd 255.255.255.255 scope global eth1
inet6 fe80::204:23ff:fec3:98d1/64 scope link
valid_lft forever preferred_lft forever
Now, maybe I''m stupid, but this seems wrong to me.
Rune
-------------------------------------------------------
Using Tomcat but need to do more? Need to support web services, security?
Get stuff done quickly with pre-integrated technology to make your job easier
Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo
http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
On Monday 08 May 2006 09:29, Rune Kock wrote:> Hi Michael > > Looking at your dump, it seems that eth0 and eth1 have the same IP-address: > > 2: eth0: <BROADCAST,MULTICAST,UP> mtu 9000 qdisc htb qlen 1000 > link/ether 00:04:23:c3:98:d0 brd ff:ff:ff:ff:ff:ff > inet 206.83.209.5/25 brd 206.83.209.127 scope global eth0 > inet6 fe80::204:23ff:fec3:98d0/64 scope link > valid_lft forever preferred_lft forever > 3: eth1: <BROADCAST,MULTICAST,UP> mtu 9000 qdisc pfifo_fast qlen 1000 > link/ether 00:04:23:c3:98:d1 brd ff:ff:ff:ff:ff:ff > inet 206.83.209.5/32 brd 255.255.255.255 scope global eth1 > inet6 fe80::204:23ff:fec3:98d1/64 scope link > valid_lft forever preferred_lft forever > > Now, maybe I''m stupid, but this seems wrong to me.This setup is often used with Proxy ARP. From my own firewall: 3: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000 link/ether aa:cc:00:00:00:03 brd ff:ff:ff:ff:ff:ff inet 206.124.146.176/32 scope global eth1 <============================= inet6 fe80::a8cc:ff:fe00:3/64 scope link valid_lft forever preferred_lft forever 4: eth3: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc htb qlen 1000 link/ether 00:a0:cc:63:66:89 brd ff:ff:ff:ff:ff:ff inet 206.124.146.176/24 brd 206.124.146.255 scope global eth3 <========= inet 206.124.146.178/24 brd 206.124.146.255 scope global secondary eth3:0 inet 206.124.146.179/24 brd 206.124.146.255 scope global secondary eth3:1 inet 206.124.146.180/24 brd 206.124.146.255 scope global secondary eth3:3 inet6 fe80::2a0:ccff:fe63:6689/64 scope link Note that in both Michael''s and in my configuration, one interface uses 255.255.255.255 as the subnet mask (/32). -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
Tom Eastep wrote:> On Monday 08 May 2006 09:29, Rune Kock wrote: > >> Hi Michael >> >> Looking at your dump, it seems that eth0 and eth1 have the same IP-address: >> >> 2: eth0: <BROADCAST,MULTICAST,UP> mtu 9000 qdisc htb qlen 1000 >> link/ether 00:04:23:c3:98:d0 brd ff:ff:ff:ff:ff:ff >> inet 206.83.209.5/25 brd 206.83.209.127 scope global eth0 >> inet6 fe80::204:23ff:fec3:98d0/64 scope link >> valid_lft forever preferred_lft forever >> 3: eth1: <BROADCAST,MULTICAST,UP> mtu 9000 qdisc pfifo_fast qlen 1000 >> link/ether 00:04:23:c3:98:d1 brd ff:ff:ff:ff:ff:ff >> inet 206.83.209.5/32 brd 255.255.255.255 scope global eth1 >> inet6 fe80::204:23ff:fec3:98d1/64 scope link >> valid_lft forever preferred_lft forever >> >> Now, maybe I''m stupid, but this seems wrong to me. >> > > This setup is often used with Proxy ARP. From my own firewall: > > 3: eth1: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc pfifo_fast qlen 1000 > link/ether aa:cc:00:00:00:03 brd ff:ff:ff:ff:ff:ff > inet 206.124.146.176/32 scope global eth1 <=============================> inet6 fe80::a8cc:ff:fe00:3/64 scope link > valid_lft forever preferred_lft forever > 4: eth3: <BROADCAST,MULTICAST,UP> mtu 1500 qdisc htb qlen 1000 > link/ether 00:a0:cc:63:66:89 brd ff:ff:ff:ff:ff:ff > inet 206.124.146.176/24 brd 206.124.146.255 scope global eth3 <=========> inet 206.124.146.178/24 brd 206.124.146.255 scope global secondary eth3:0 > inet 206.124.146.179/24 brd 206.124.146.255 scope global secondary eth3:1 > inet 206.124.146.180/24 brd 206.124.146.255 scope global secondary eth3:3 > inet6 fe80::2a0:ccff:fe63:6689/64 scope link > > Note that in both Michael''s and in my configuration, one interface uses > 255.255.255.255 as the subnet mask (/32). > > -Tom >Yeah - the documentation recommended that setup for proxyarp and I was happy to oblige to save IPs. ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
On Monday 08 May 2006 10:01, Rune Kock wrote:> On 5/8/06, Tom Eastep <teastep@shorewall.net> wrote: > > On Monday 08 May 2006 09:29, Rune Kock wrote: > > > Hi Michael > > > > > > Looking at your dump, it seems that eth0 and eth1 have the same > > > IP-address: > > > > > > 2: eth0: <BROADCAST,MULTICAST,UP> mtu 9000 qdisc htb qlen 1000 > > > link/ether 00:04:23:c3:98:d0 brd ff:ff:ff:ff:ff:ff > > > inet 206.83.209.5/25 brd 206.83.209.127 scope global eth0 > > > inet6 fe80::204:23ff:fec3:98d0/64 scope link > > > valid_lft forever preferred_lft forever > > > 3: eth1: <BROADCAST,MULTICAST,UP> mtu 9000 qdisc pfifo_fast qlen 1000 > > > link/ether 00:04:23:c3:98:d1 brd ff:ff:ff:ff:ff:ff > > > inet 206.83.209.5/32 brd 255.255.255.255 scope global eth1 > > > inet6 fe80::204:23ff:fec3:98d1/64 scope link > > > valid_lft forever preferred_lft forever > > > > > > Now, maybe I''m stupid, but this seems wrong to me. > > > > This setup is often used with Proxy ARP. From my own firewall: > > [snip] > > Okay, I never used Proxy ARP myself, so I didn''t recognize it. > > Another thought: could it be that Michael''s unusually large MTU (9000) > somehow messes up the HTB-timings? Even though 9000 is not exactly > 1500*4, it is close.Interesting theory. It''s hard to conclude much from that dump since it is clear that no packet classification was occuring and that all traffic was using the default class (all marked traffic was local and/or input traffic). -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
Tom Eastep wrote:> On Monday 08 May 2006 10:01, Rune Kock wrote: > >> On 5/8/06, Tom Eastep <teastep@shorewall.net> wrote: >> >>> On Monday 08 May 2006 09:29, Rune Kock wrote: >>> >>>> Hi Michael >>>> >>>> Looking at your dump, it seems that eth0 and eth1 have the same >>>> IP-address: >>>> >>>> 2: eth0: <BROADCAST,MULTICAST,UP> mtu 9000 qdisc htb qlen 1000 >>>> link/ether 00:04:23:c3:98:d0 brd ff:ff:ff:ff:ff:ff >>>> inet 206.83.209.5/25 brd 206.83.209.127 scope global eth0 >>>> inet6 fe80::204:23ff:fec3:98d0/64 scope link >>>> valid_lft forever preferred_lft forever >>>> 3: eth1: <BROADCAST,MULTICAST,UP> mtu 9000 qdisc pfifo_fast qlen 1000 >>>> link/ether 00:04:23:c3:98:d1 brd ff:ff:ff:ff:ff:ff >>>> inet 206.83.209.5/32 brd 255.255.255.255 scope global eth1 >>>> inet6 fe80::204:23ff:fec3:98d1/64 scope link >>>> valid_lft forever preferred_lft forever >>>> >>>> Now, maybe I''m stupid, but this seems wrong to me. >>>> >>> This setup is often used with Proxy ARP. From my own firewall: >>> >> [snip] >> >> Okay, I never used Proxy ARP myself, so I didn''t recognize it. >> >> Another thought: could it be that Michael''s unusually large MTU (9000) >> somehow messes up the HTB-timings? Even though 9000 is not exactly >> 1500*4, it is close. >> > > Interesting theory. > > It''s hard to conclude much from that dump since it is clear that no packet > classification was occuring and that all traffic was using the default > class > (all marked traffic was local and/or input traffic). > > -Tom >I actually already tested that - same result at 9000 MTU. ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
Tom Eastep wrote:> On Monday 08 May 2006 10:01, Rune Kock wrote: > >> On 5/8/06, Tom Eastep <teastep@shorewall.net> wrote: >> >>> On Monday 08 May 2006 09:29, Rune Kock wrote: >>> >>>> Hi Michael >>>> >>>> Looking at your dump, it seems that eth0 and eth1 have the same >>>> IP-address: >>>> >>>> 2: eth0: <BROADCAST,MULTICAST,UP> mtu 9000 qdisc htb qlen 1000 >>>> link/ether 00:04:23:c3:98:d0 brd ff:ff:ff:ff:ff:ff >>>> inet 206.83.209.5/25 brd 206.83.209.127 scope global eth0 >>>> inet6 fe80::204:23ff:fec3:98d0/64 scope link >>>> valid_lft forever preferred_lft forever >>>> 3: eth1: <BROADCAST,MULTICAST,UP> mtu 9000 qdisc pfifo_fast qlen 1000 >>>> link/ether 00:04:23:c3:98:d1 brd ff:ff:ff:ff:ff:ff >>>> inet 206.83.209.5/32 brd 255.255.255.255 scope global eth1 >>>> inet6 fe80::204:23ff:fec3:98d1/64 scope link >>>> valid_lft forever preferred_lft forever >>>> >>>> Now, maybe I''m stupid, but this seems wrong to me. >>>> >>> This setup is often used with Proxy ARP. From my own firewall: >>> >> [snip] >> >> Okay, I never used Proxy ARP myself, so I didn''t recognize it. >> >> Another thought: could it be that Michael''s unusually large MTU (9000) >> somehow messes up the HTB-timings? Even though 9000 is not exactly >> 1500*4, it is close. >> > > Interesting theory. > > It''s hard to conclude much from that dump since it is clear that no packet > classification was occuring and that all traffic was using the default > class > (all marked traffic was local and/or input traffic). > > -Tom >Doh - I ment same result at 1500 :-) ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
On 5/8/06, Michael McCallister <mikemc-shorewall@contactdesigns.com> wrote:> On Monday 08 May 2006 10:01, Rune Kock wrote: >> Another thought: could it be that Michael''s unusually large MTU (9000) >> somehow messes up the HTB-timings? Even though 9000 is not exactly >> 1500*4, it is close. >> > I actually already tested that - same result at 9000 MTU. > Doh - I ment same result at 1500 :-)I''m still pursuing this line; maybe you missed something in your test. From your original dump: | class htb 1:14 parent 1:1 leaf 14: prio 4 quantum 13107 rate 1000Kbit | ceil 1000Kbit burst 2099b/8 mpu 0b overhead 0b cburst 2099b/8 mpu | 0b overhead 0b level 0 | Sent 170069819 bytes 25182 pkt (dropped 0, overlimits 0 requeues 0) | rate 0bit 0pps backlog 0b 0p requeues 0 | lended: 25182 borrowed: 0 giants: 42690 | tokens: 16876 ctokens: 16876 Then from the HTB manual: | giants is number of packets larger than mtu set in tc command. HTB will | work with these but rates will not be accurate at all. Add mtu to your tc | (defaults to 1600 bytes). It seems that Shorewall may need to specify the MTU when setting up HTB if the interface has an unusually large MTU. Rune ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
Rune Kock wrote:> > Then from the HTB manual: >Rune, Where are you finding the HTB manual? Divik''s site appears to be off-line. Thanks, -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
> Where are you finding the HTB manual? Divik''s site appears to be off-line.I used the Google cache, http://66.249.93.104/search?q=cache:_0aXkErEjrEJ:luxik.cdi.cz/~devik/qos/htb/manual/userg.htm&hl=da&gl=dk&ct=clnk&cd=1&client=firefox-a Hopefully, the site will be up again soon. ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
Rune Kock wrote:>> Where are you finding the HTB manual? Divik''s site appears to be >> off-line. > > > I used the Google cache, > http://66.249.93.104/search?q=cache:_0aXkErEjrEJ:luxik.cdi.cz/~devik/qos/htb/manual/userg.htm&hl=da&gl=dk&ct=clnk&cd=1&client=firefox-a > > > Hopefully, the site will be up again soon.Thanks. -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
Rune Kock wrote:> On 5/8/06, Michael McCallister <mikemc-shorewall@contactdesigns.com> > wrote: >> On Monday 08 May 2006 10:01, Rune Kock wrote: >>> Another thought: could it be that Michael''s unusually large MTU (9000) >>> somehow messes up the HTB-timings? Even though 9000 is not exactly >>> 1500*4, it is close. >>> >> I actually already tested that - same result at 9000 MTU. >> Doh - I ment same result at 1500 :-) > > I''m still pursuing this line; maybe you missed something in your test. > > From your original dump: > > | class htb 1:14 parent 1:1 leaf 14: prio 4 quantum 13107 rate 1000Kbit > | ceil 1000Kbit burst 2099b/8 mpu 0b overhead 0b cburst 2099b/8 mpu > | 0b overhead 0b level 0 > | Sent 170069819 bytes 25182 pkt (dropped 0, overlimits 0 requeues 0) > | rate 0bit 0pps backlog 0b 0p requeues 0 > | lended: 25182 borrowed: 0 giants: 42690 > | tokens: 16876 ctokens: 16876 > > Then from the HTB manual: > > | giants is number of packets larger than mtu set in tc command. HTB will > | work with these but rates will not be accurate at all. Add mtu to > your tc > | (defaults to 1600 bytes). > > It seems that Shorewall may need to specify the MTU when setting up > HTB if the interface has an unusually large MTU. > > > Rune >Hi Rune, I tested this again using "ifconfig eth0 mtu 1500" (I also did eth1):> [root@io ~]# ifconfig > eth0 Link encap:Ethernet HWaddr 00:04:23:C3:98:D0 > inet addr:206.83.209.5 Bcast:206.83.209.127 > Mask:255.255.255.128 > inet6 addr: fe80::204:23ff:fec3:98d0/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:438517 errors:0 dropped:0 overruns:0 frame:0 > TX packets:383597 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:35001248 (33.3 MiB) TX bytes:265577538 (253.2 MiB) > Base address:0xe800 Memory:feac0000-feae0000 > > eth1 Link encap:Ethernet HWaddr 00:04:23:C3:98:D1 > inet addr:206.83.209.5 Bcast:255.255.255.255 > Mask:255.255.255.255 > inet6 addr: fe80::204:23ff:fec3:98d1/64 Scope:Link > UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 > RX packets:363700 errors:0 dropped:0 overruns:0 frame:0 > TX packets:198734 errors:0 dropped:0 overruns:0 carrier:0 > collisions:0 txqueuelen:1000 > RX bytes:242533543 (231.2 MiB) TX bytes:17413689 (16.6 MiB) > Base address:0xec00 Memory:feae0000-feb00000According to the class and a tcdevices of 1000kbit, it should transfer over sftp at 333kbit. But Winscp over here is reporting 150-165/KBs - still about four times more than it should be transferring (at least according to the conversion calculator I use: http://www.speedguide.net/conversion.php). Does something else need to be done other than setting mtu via ifconfig? I can reboot with those settings set to load at boot if that helps any. Also, if that MTU documentation you are referring to is correct, then why would servers behind the firewall have correct transfer speeds? Their packets have to go through the firewall NICs which have 9000 MTU settings - so wouldn''t their speeds be incorrect as well by this reasoning? Also, not that it should matter - but I am running a x86_64 kernel. Michael ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
Michael McCallister wrote:> Also, if that MTU > documentation you are referring to is correct, then why would servers > behind the firewall have correct transfer speeds? Their packets have to > go through the firewall NICs which have 9000 MTU settings - so wouldn''t > their speeds be incorrect as well by this reasoning?What MTU setting is configured on the servers? -Tom -- Tom Eastep \ Nothing is foolproof to a sufficiently talented fool Shoreline, \ http://shorewall.net Washington USA \ teastep@shorewall.net PGP Public Key \ https://lists.shorewall.net/teastep.pgp.key
Rune Kock wrote the following on 05/09/2006 09:38 AM:> Suddenly, the mailing list rejects my message. So here it is directly > to you. > > ---------- Forwarded message ---------- > From: Rune Kock <rune.kock@gmail.com> > Date: May 9, 2006 6:31 PM > Subject: Re: [Shorewall-users] traffic shaping problems > To: shorewall-users@lists.sourceforge.net > > > On 5/9/06, Michael McCallister <mikemc-shorewall@contactdesigns.com> > wrote: > >> I tested this again using "ifconfig eth0 mtu 1500" (I also did eth1): > > [snip] > >> Does something else need to >> be done other than setting mtu via ifconfig? I can reboot with those >> settings set to load at boot if that helps any. > > > I haven''t tried changing MTU myself, so I cannot tell whether ifconfig > works. Also, I don''t have Linux on this machine (shame on me...) so I > have no way of testing at the moment. > > If you like, you can use a packet sniffer like Ethereal to check the > actual size of the packets. > >> Also, if that MTU >> documentation you are referring to is correct, then why would servers >> behind the firewall have correct transfer speeds? Their packets have to >> go through the firewall NICs which have 9000 MTU settings - so wouldn''t >> their speeds be incorrect as well by this reasoning? > > > As I read it, the miscalculations will only occur if the actual > packets are big. If your servers have their MTU at 1500, that will be > the size of your packets. > >> Also, not that it >> should matter - but I am running a x86_64 kernel. > > > One never knows where bugs might creep in :) > > > RuneHi Rune, Yeah, none of the NICs behind the firewall support MTUs greater than 1500. The Intel NICs on the firewall (eth0 and eth1) do support larger MTUs. Also, the switches that they all run on support "jumbo frames" which is why I configured MTUs of 9000 (I also read a bunch of stuff that said 1500 MTUs limit bandwidth on gigabit ethernet: http://www.google.com/search?q=jumbo+frames). Anyways, I rebooted the machine after setting MTUs back to 1500 for all NICs in /etc/sysconfig. Things still download faster than ceiling specified in tcdevices (150-165/KBs again). Posted dump of current settings here: http://caffeine.contactdesigns.com/~jolt/shorewalldump2.txt I currently have tcdevices at much lower badwidth settings (1000kbit) than I will need later this month when that rack goes into production, but I found that 1000kbit is good for testing. Michael ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
On 5/9/06, Michael McCallister <mikemc-shorewall@contactdesigns.com> wrote:> Yeah, none of the NICs behind the firewall support MTUs greater than > 1500. The Intel NICs on the firewall (eth0 and eth1) do support larger > MTUs. Also, the switches that they all run on support "jumbo frames" > which is why I configured MTUs of 9000 (I also read a bunch of stuff > that said 1500 MTUs limit bandwidth on gigabit ethernet: > http://www.google.com/search?q=jumbo+frames).Well, I agree that a big MTU is a good idea for pure LAN-traffic on gigabit. If the traffic is going to the internet, a big packet is unlikely to get all the way through, and in that case it is slightly faster to use a small packet to begin with.> Anyways, I rebooted the machine after setting MTUs back to 1500 for all > NICs in /etc/sysconfig. Things still download faster than ceiling > specified in tcdevices (150-165/KBs again). Posted dump of current > settings here: > http://caffeine.contactdesigns.com/~jolt/shorewalldump2.txt I currently > have tcdevices at much lower badwidth settings (1000kbit) than I will > need later this month when that rack goes into production, but I found > that 1000kbit is good for testing.Your dump still shows a lot of "giants": class htb 1:1 root rate 1000Kbit ceil 1000Kbit burst 2099b/8 mpu 0b overhead 0b cburst 2099b/8 mpu 0b overhead 0b level 7 Sent 80746108 bytes 11007 pkt (dropped 0, overlimits 0 requeues 0) rate 1784bit 1pps backlog 0b 0p requeues 0 lended: 0 borrowed: 0 giants: 17268 tokens: 15762 ctokens: 15762 Maybe your nic uses a 9k MTU no matter what you specify???? I think it is time for the packet sniffer.... Rune ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642
Rune Kock wrote the following on 05/09/2006 10:39 AM:> On 5/9/06, Michael McCallister <mikemc-shorewall@contactdesigns.com> > wrote: > >> Yeah, none of the NICs behind the firewall support MTUs greater than >> 1500. The Intel NICs on the firewall (eth0 and eth1) do support larger >> MTUs. Also, the switches that they all run on support "jumbo frames" >> which is why I configured MTUs of 9000 (I also read a bunch of stuff >> that said 1500 MTUs limit bandwidth on gigabit ethernet: >> http://www.google.com/search?q=jumbo+frames). > > > Well, I agree that a big MTU is a good idea for pure LAN-traffic on > gigabit. If the traffic is going to the internet, a big packet is > unlikely to get all the way through, and in that case it is slightly > faster to use a small packet to begin with. > >> Anyways, I rebooted the machine after setting MTUs back to 1500 for all >> NICs in /etc/sysconfig. Things still download faster than ceiling >> specified in tcdevices (150-165/KBs again). Posted dump of current >> settings here: >> http://caffeine.contactdesigns.com/~jolt/shorewalldump2.txt I currently >> have tcdevices at much lower badwidth settings (1000kbit) than I will >> need later this month when that rack goes into production, but I found >> that 1000kbit is good for testing. > > > Your dump still shows a lot of "giants": > > class htb 1:1 root rate 1000Kbit ceil 1000Kbit burst 2099b/8 mpu 0b > overhead 0b cburst 2099b/8 mpu 0b overhead 0b level 7 > Sent 80746108 bytes 11007 pkt (dropped 0, overlimits 0 requeues 0) > rate 1784bit 1pps backlog 0b 0p requeues 0 > lended: 0 borrowed: 0 giants: 17268 > tokens: 15762 ctokens: 15762 > > Maybe your nic uses a 9k MTU no matter what you specify???? I think > it is time for the packet sniffer.... > > > RuneHow do you use ethereal? I managed to get it installed. After running "tethereal", I get stuff like this:> 18.828561 206.83.209.5 -> 64.242.33.55 SSH Encrypted response packet > len=240 > 18.844783 64.242.33.55 -> 206.83.209.5 TCP 47063 > ssh [ACK] Seq=192 > Ack=145744 Win=8616 Len=0 TSV=9428295 TSER=1376375 > 18.844803 206.83.209.5 -> 64.242.33.55 SSH Encrypted response packet > len=240 > 18.861524 64.242.33.55 -> 206.83.209.5 SSH Encrypted request packet > len=48 > 18.861543 206.83.209.5 -> 64.242.33.55 TCP ssh > 47063 [ACK] > Seq=147216 Ack=240 Win=4968 Len=0 TSV=1376440 TSER=9428311man page makes no mention of mtu. Michael ------------------------------------------------------- Using Tomcat but need to do more? Need to support web services, security? Get stuff done quickly with pre-integrated technology to make your job easier Download IBM WebSphere Application Server v.1.0.1 based on Apache Geronimo http://sel.as-us.falkag.net/sel?cmd=lnk&kid=120709&bid=263057&dat=121642