olivier.monaco@free.fr
2013-Dec-03 22:03 UTC
Multiple ISP + traffic shapping = poor download speed
Hello,
Thanks for the great Shorewall which has replaced my hard to maintain home-made
scripts.
First, what works.
Our local network is 10.48.X.X with multiple vlan, each on a dedicated
interface. We use Shorewall 4.4.11 from Debian Squeeze.
We have a 2 ISP:
- isp1 : an optical fiber provider with 10 Mbps.
- isp2 : a DSL provider with 15Mbits/1Mbits.
We use isp2 as the default outgoing provider. The isp1 provider is used for
"critical" services (SSH...) and for incoming connections (VPN...).
Our interfaces file :
=======================isp1 eth0 detect
logmartians,nosmurfs,routefilter=0,tcpflags
isp2 eth1 detect logmartians,nosmurfs,routefilter,tcpflags
=======================
Here is our providers file:
=======================#NAME NUMBER MARK DUPLICATE INTERFACE
GATEWAY OPTIONS COPY
isp1 1 0x100 - eth1 37.X.X.X track,loose
-
isp2 2 0x200 - eth2 217.X.X.X track,balance
-
=======================
Here is an extract of our tcrules file:
=======================######################################################################################################################
#MARK SOURCE DEST PROTO DEST SOURCE USER TEST
LENGTH TOS CONNBYTES HELPER
# PORT(S) PORT(S)
# ISP1 DNS => ISP1
256 0.0.0.0/0 37.X.X.X
256 $FW 37.X.X.X
# ISP2 DNS => ISP2
512 0.0.0.0/0 127.X.X.X
512 $FW 127.X.X.X
# Google DNS => ISP1
256 0.0.0.0/0 8.8.8.8,8.8.4.4
256 $FW 8.8.8.8,8.8.4.4
# VPN IPsec (out) => ISP1
256 0.0.0.0/0 0.0.0.0/0 udp 500,4500
256 $FW 0.0.0.0/0 udp 500,4500
# Force one host to ISP1
256 10.48.1.10 0.0.0.0/0
# Force all SSH to ISP1
256 0.0.0.0/0 0.0.0.0/0 tcp 22
256 $FW 0.0.0.0/0 tcp 22
=======================
Yesterday we added VoIP. To do so, we force traffic from our Asterisk server to
go throw ISP1 with a dedicated public IP and force the traffic from this
dedicated public IP to go to Asterisk server (with IP filtering for security).
This works too.
Now, my problem is to put QoS (using TC_ENABLED=Internal). I try many
configuration but always have the same problem: once the isp1 interface is
listed in tcdevices, we have poor download speed. Even with/without other TC
configuration.
Here is our tcdevices file:
=======================#NUMBER: IN-BANDWITH OUT-BANDWIDTH OPTIONS REDIRECTED
#INTERFACE INTERFACES
1:isp1 10240kbit 10240kbit
=======================
We use an external server to test download speed with IP 5.X.X.X so we added in
tcrules:
=======================256 0.0.0.0/0 5.X.X.X
$FW 0.0.0.0/0 5.X.X.X
=======================
The results are:
- without isp1 in tcdevices => more than 1MB/s (bytes measured with wget
command)
- with isp1 in tcdevices => less than 300 kB/s
If I change bandwidth of isp1 to something more than 70000kbit, all goes
right... Other lower value have the same problem but with different download
speed (seems proportional to the interface speed).
Here is a result of the following command: tc -s -d class show dev isp1
======================== class htb 1:1 root rate 10240Kbit ceil 10240Kbit burst
1598b/8 mpu 0b overhead 0b cburst 1598b/8 mpu 0b overhead 0b level 7
Sent 1111091 bytes 11680 pkt (dropped 0, overlimits 0 requeues 0)
rate 83656bit 124pps backlog 0b 0p requeues 0
lended: 0 borrowed: 0 giants: 0
tokens: 17781 ctokens: 17781
=======================
Rates seems to be OK.
Have someone the same problem?
Regards,
Olivier Monaco
------------------------------------------------------------------------------
Sponsored by Intel(R) XDK
Develop, test and display web and hybrid apps with a single code base.
Download it for free now!
http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
Tom Eastep
2013-Dec-03 23:56 UTC
Re: Multiple ISP + traffic shapping = poor download speed
On 12/3/2013 2:03 PM, olivier.monaco@free.fr wrote:> Hello, > > Thanks for the great Shorewall which has replaced my hard to maintain home-made scripts. > > First, what works. > > Our local network is 10.48.X.X with multiple vlan, each on a dedicated interface. We use Shorewall 4.4.11 from Debian Squeeze. > > We have a 2 ISP: > - isp1 : an optical fiber provider with 10 Mbps. > - isp2 : a DSL provider with 15Mbits/1Mbits. > > We use isp2 as the default outgoing provider. The isp1 provider is used for "critical" services (SSH...) and for incoming connections (VPN...). > > Our interfaces file : > =======================> isp1 eth0 detect logmartians,nosmurfs,routefilter=0,tcpflags > isp2 eth1 detect logmartians,nosmurfs,routefilter,tcpflags > =======================> > Here is our providers file: > =======================> #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY > isp1 1 0x100 - eth1 37.X.X.X track,loose - > isp2 2 0x200 - eth2 217.X.X.X track,balance - > =======================> > Here is an extract of our tcrules file: > =======================> ###################################################################################################################### > #MARK SOURCE DEST PROTO DEST SOURCE USER TEST LENGTH TOS CONNBYTES HELPER > # PORT(S) PORT(S) > > # ISP1 DNS => ISP1 > 256 0.0.0.0/0 37.X.X.X > 256 $FW 37.X.X.X > > # ISP2 DNS => ISP2 > 512 0.0.0.0/0 127.X.X.X > 512 $FW 127.X.X.X > > # Google DNS => ISP1 > 256 0.0.0.0/0 8.8.8.8,8.8.4.4 > 256 $FW 8.8.8.8,8.8.4.4 > > # VPN IPsec (out) => ISP1 > 256 0.0.0.0/0 0.0.0.0/0 udp 500,4500 > 256 $FW 0.0.0.0/0 udp 500,4500 > > # Force one host to ISP1 > 256 10.48.1.10 0.0.0.0/0 > > # Force all SSH to ISP1 > 256 0.0.0.0/0 0.0.0.0/0 tcp 22 > 256 $FW 0.0.0.0/0 tcp 22 > =======================> > Yesterday we added VoIP. To do so, we force traffic from our Asterisk server to go throw ISP1 with a dedicated public IP and force the traffic from this dedicated public IP to go to Asterisk server (with IP filtering for security). This works too. > > Now, my problem is to put QoS (using TC_ENABLED=Internal). I try many configuration but always have the same problem: once the isp1 interface is listed in tcdevices, we have poor download speed. Even with/without other TC configuration. > > Here is our tcdevices file: > =======================> #NUMBER: IN-BANDWITH OUT-BANDWIDTH OPTIONS REDIRECTED > #INTERFACE INTERFACES > 1:isp1 10240kbit 10240kbit > =======================> > We use an external server to test download speed with IP 5.X.X.X so we added in tcrules: > =======================> 256 0.0.0.0/0 5.X.X.X > $FW 0.0.0.0/0 5.X.X.X > =======================> > The results are: > - without isp1 in tcdevices => more than 1MB/s (bytes measured with wget command) > - with isp1 in tcdevices => less than 300 kB/s > > If I change bandwidth of isp1 to something more than 70000kbit, all goes right... Other lower value have the same problem but with different download speed (seems proportional to the interface speed). > > Here is a result of the following command: tc -s -d class show dev isp1 ======================== class htb 1:1 root rate 10240Kbit ceil 10240Kbit burst 1598b/8 mpu 0b overhead 0b cburst 1598b/8 mpu 0b overhead 0b level 7 > Sent 1111091 bytes 11680 pkt (dropped 0, overlimits 0 requeues 0) > rate 83656bit 124pps backlog 0b 0p requeues 0 > lended: 0 borrowed: 0 giants: 0 > tokens: 17781 ctokens: 17781 > =======================> > Rates seems to be OK. > > Have someone the same problem? >Sounds like Shorewall FAQ 97a. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Sponsored by Intel(R) XDK Develop, test and display web and hybrid apps with a single code base. Download it for free now! http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
olivier.monaco@free.fr
2013-Dec-04 19:59 UTC
Re: Multiple ISP + traffic shapping = poor download speed
It's not. # ethtool -k eth1 Offload parameters for eth1: rx-checksumming: on tx-checksumming: on scatter-gather: on tcp-segmentation-offload: off udp-fragmentation-offload: off generic-segmentation-offload: off generic-receive-offload: off large-receive-offload: off ntuple-filters: off receive-hashing: off -Olivier ----- Mail original ----- De: "Tom Eastep" <teastep@shorewall.net> À: "Shorewall Users" <shorewall-users@lists.sourceforge.net> Envoyé: Mercredi 4 Décembre 2013 00:56:39 Objet: Re: [Shorewall-users] Multiple ISP + traffic shapping = poor download speed On 12/3/2013 2:03 PM, olivier.monaco@free.fr wrote:> Hello, > > Thanks for the great Shorewall which has replaced my hard to maintain home-made scripts. > > First, what works. > > Our local network is 10.48.X.X with multiple vlan, each on a dedicated interface. We use Shorewall 4.4.11 from Debian Squeeze. > > We have a 2 ISP: > - isp1 : an optical fiber provider with 10 Mbps. > - isp2 : a DSL provider with 15Mbits/1Mbits. > > We use isp2 as the default outgoing provider. The isp1 provider is used for "critical" services (SSH...) and for incoming connections (VPN...). > > Our interfaces file : > =======================> isp1 eth0 detect logmartians,nosmurfs,routefilter=0,tcpflags > isp2 eth1 detect logmartians,nosmurfs,routefilter,tcpflags > =======================> > Here is our providers file: > =======================> #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY OPTIONS COPY > isp1 1 0x100 - eth1 37.X.X.X track,loose - > isp2 2 0x200 - eth2 217.X.X.X track,balance - > =======================> > Here is an extract of our tcrules file: > =======================> ###################################################################################################################### > #MARK SOURCE DEST PROTO DEST SOURCE USER TEST LENGTH TOS CONNBYTES HELPER > # PORT(S) PORT(S) > > # ISP1 DNS => ISP1 > 256 0.0.0.0/0 37.X.X.X > 256 $FW 37.X.X.X > > # ISP2 DNS => ISP2 > 512 0.0.0.0/0 127.X.X.X > 512 $FW 127.X.X.X > > # Google DNS => ISP1 > 256 0.0.0.0/0 8.8.8.8,8.8.4.4 > 256 $FW 8.8.8.8,8.8.4.4 > > # VPN IPsec (out) => ISP1 > 256 0.0.0.0/0 0.0.0.0/0 udp 500,4500 > 256 $FW 0.0.0.0/0 udp 500,4500 > > # Force one host to ISP1 > 256 10.48.1.10 0.0.0.0/0 > > # Force all SSH to ISP1 > 256 0.0.0.0/0 0.0.0.0/0 tcp 22 > 256 $FW 0.0.0.0/0 tcp 22 > =======================> > Yesterday we added VoIP. To do so, we force traffic from our Asterisk server to go throw ISP1 with a dedicated public IP and force the traffic from this dedicated public IP to go to Asterisk server (with IP filtering for security). This works too. > > Now, my problem is to put QoS (using TC_ENABLED=Internal). I try many configuration but always have the same problem: once the isp1 interface is listed in tcdevices, we have poor download speed. Even with/without other TC configuration. > > Here is our tcdevices file: > =======================> #NUMBER: IN-BANDWITH OUT-BANDWIDTH OPTIONS REDIRECTED > #INTERFACE INTERFACES > 1:isp1 10240kbit 10240kbit > =======================> > We use an external server to test download speed with IP 5.X.X.X so we added in tcrules: > =======================> 256 0.0.0.0/0 5.X.X.X > $FW 0.0.0.0/0 5.X.X.X > =======================> > The results are: > - without isp1 in tcdevices => more than 1MB/s (bytes measured with wget command) > - with isp1 in tcdevices => less than 300 kB/s > > If I change bandwidth of isp1 to something more than 70000kbit, all goes right... Other lower value have the same problem but with different download speed (seems proportional to the interface speed). > > Here is a result of the following command: tc -s -d class show dev isp1 ======================== class htb 1:1 root rate 10240Kbit ceil 10240Kbit burst 1598b/8 mpu 0b overhead 0b cburst 1598b/8 mpu 0b overhead 0b level 7 > Sent 1111091 bytes 11680 pkt (dropped 0, overlimits 0 requeues 0) > rate 83656bit 124pps backlog 0b 0p requeues 0 > lended: 0 borrowed: 0 giants: 0 > tokens: 17781 ctokens: 17781 > =======================> > Rates seems to be OK. > > Have someone the same problem? >Sounds like Shorewall FAQ 97a. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Sponsored by Intel(R) XDK Develop, test and display web and hybrid apps with a single code base. Download it for free now! http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
Simon Matter
2013-Dec-05 06:59 UTC
Re: Multiple ISP + traffic shapping = poor download speed
> It''s not. > > # ethtool -k eth1 > Offload parameters for eth1: > rx-checksumming: on > tx-checksumming: on > scatter-gather: on > tcp-segmentation-offload: off > udp-fragmentation-offload: off > generic-segmentation-offload: off > generic-receive-offload: off > large-receive-offload: off > ntuple-filters: off > receive-hashing: offAnd what''s on eth0 and eth2? What I don''t understand is your interfaces file, where you have isp1 on eth0 and isp2 on eth1, while in another place you have isp1 -> eth1 and isp2 -> eth2. Is this all correct? Simon> > -Olivier > ----- Mail original ----- > De: "Tom Eastep" <teastep@shorewall.net> > Ã: "Shorewall Users" <shorewall-users@lists.sourceforge.net> > Envoyé: Mercredi 4 Décembre 2013 00:56:39 > Objet: Re: [Shorewall-users] Multiple ISP + traffic shapping > poor download speed > > On 12/3/2013 2:03 PM, olivier.monaco@free.fr wrote: >> Hello, >> >> Thanks for the great Shorewall which has replaced my hard to maintain >> home-made scripts. >> >> First, what works. >> >> Our local network is 10.48.X.X with multiple vlan, each on a dedicated >> interface. We use Shorewall 4.4.11 from Debian Squeeze. >> >> We have a 2 ISP: >> - isp1 : an optical fiber provider with 10 Mbps. >> - isp2 : a DSL provider with 15Mbits/1Mbits. >> >> We use isp2 as the default outgoing provider. The isp1 provider is used >> for "critical" services (SSH...) and for incoming connections (VPN...). >> >> Our interfaces file : >> =======================>> isp1 eth0 detect >> logmartians,nosmurfs,routefilter=0,tcpflags >> isp2 eth1 detect >> logmartians,nosmurfs,routefilter,tcpflags >> =======================>> >> Here is our providers file: >> =======================>> #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY >> OPTIONS COPY >> isp1 1 0x100 - eth1 37.X.X.X >> track,loose - >> isp2 2 0x200 - eth2 217.X.X.X >> track,balance - >> =======================>> >> Here is an extract of our tcrules file: >> =======================>> ###################################################################################################################### >> #MARK SOURCE DEST PROTO DEST SOURCE USER >> TEST LENGTH TOS CONNBYTES HELPER >> # PORT(S) PORT(S) >> >> # ISP1 DNS => ISP1 >> 256 0.0.0.0/0 37.X.X.X >> 256 $FW 37.X.X.X >> >> # ISP2 DNS => ISP2 >> 512 0.0.0.0/0 127.X.X.X >> 512 $FW 127.X.X.X >> >> # Google DNS => ISP1 >> 256 0.0.0.0/0 8.8.8.8,8.8.4.4 >> 256 $FW 8.8.8.8,8.8.4.4 >> >> # VPN IPsec (out) => ISP1 >> 256 0.0.0.0/0 0.0.0.0/0 udp 500,4500 >> 256 $FW 0.0.0.0/0 udp 500,4500 >> >> # Force one host to ISP1 >> 256 10.48.1.10 0.0.0.0/0 >> >> # Force all SSH to ISP1 >> 256 0.0.0.0/0 0.0.0.0/0 tcp 22 >> 256 $FW 0.0.0.0/0 tcp 22 >> =======================>> >> Yesterday we added VoIP. To do so, we force traffic from our Asterisk >> server to go throw ISP1 with a dedicated public IP and force the traffic >> from this dedicated public IP to go to Asterisk server (with IP >> filtering for security). This works too. >> >> Now, my problem is to put QoS (using TC_ENABLED=Internal). I try many >> configuration but always have the same problem: once the isp1 interface >> is listed in tcdevices, we have poor download speed. Even with/without >> other TC configuration. >> >> Here is our tcdevices file: >> =======================>> #NUMBER: IN-BANDWITH OUT-BANDWIDTH OPTIONS REDIRECTED >> #INTERFACE INTERFACES >> 1:isp1 10240kbit 10240kbit >> =======================>> >> We use an external server to test download speed with IP 5.X.X.X so we >> added in tcrules: >> =======================>> 256 0.0.0.0/0 5.X.X.X >> $FW 0.0.0.0/0 5.X.X.X >> =======================>> >> The results are: >> - without isp1 in tcdevices => more than 1MB/s (bytes measured with wget >> command) >> - with isp1 in tcdevices => less than 300 kB/s >> >> If I change bandwidth of isp1 to something more than 70000kbit, all goes >> right... Other lower value have the same problem but with different >> download speed (seems proportional to the interface speed). >> >> Here is a result of the following command: tc -s -d class show dev isp1 >> ======================== class htb 1:1 root rate 10240Kbit ceil >> 10240Kbit burst 1598b/8 mpu 0b overhead 0b cburst 1598b/8 mpu 0b >> overhead 0b level 7 >> Sent 1111091 bytes 11680 pkt (dropped 0, overlimits 0 requeues 0) >> rate 83656bit 124pps backlog 0b 0p requeues 0 >> lended: 0 borrowed: 0 giants: 0 >> tokens: 17781 ctokens: 17781 >> =======================>> >> Rates seems to be OK. >> >> Have someone the same problem? >> > > Sounds like Shorewall FAQ 97a. > > -Tom > -- > Tom Eastep \ When I die, I want to go like my Grandfather who > Shoreline, \ died peacefully in his sleep. Not screaming like > Washington, USA \ all of the passengers in his car > http://shorewall.net \________________________________________________ > > > ------------------------------------------------------------------------------ > Sponsored by Intel(R) XDK > Develop, test and display web and hybrid apps with a single code base. > Download it for free now! > http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk > _______________________________________________ > Shorewall-users mailing list > Shorewall-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/shorewall-users >------------------------------------------------------------------------------ Sponsored by Intel(R) XDK Develop, test and display web and hybrid apps with a single code base. Download it for free now! http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk
olivier.monaco@free.fr
2013-Dec-05 19:55 UTC
Re: Multiple ISP + traffic shapping = poor download speed
Hello, My first message contains a simplified setup. Our setup is a bit more complicated. We have 2 companies. The firewall have 4 physical interfaces and routes traffic between 5 internal networks and 3 ISPs. We use 7 VLANs, some are tagged by the server (and use eth0 as raw interface), some are tagged by our switch. The 5 internal networks are: - VLAN2, 10.48.2.254/24: network management. - VLAN3, 10.48.3.254/24: visitor access. - VLAN4, 10.48.4.254/24: company 1. - VLAN5, 10.48.5.254/24: company 2. - VLAN10, 10.48.10.254/24: voip. - VLAN50, 213.X.X.X/30: ISP3. - VLAN51, 10.48.51.2/24: ISP2. The 3 ISPs are: - ISP1 through eth0: optical fiber, 10Mbits, 37.X.X.X/29. - ISP2 through VLAN51: adsl, 15Mbits/1Mbits, connected to the ISP router through network 10.48.51.2/24, public IP is 217.X.X.X. - ISP3 through VLAN50: sdsl, 2Mbits, will disappear at the end of this month. So, here are all our interfaces: ^ Iface ^ Address ^ Description ^ VLAN | eth0 | 10.48.2.254/24 | Network management | 2, tagged by our switch | eth1 | 37.X.X.X/29 | ISP 1 | | eth2 | 10.48.4.254/24 | Company 1 | 4, tagged by our switch | eth3 | 10.48.5.254/24 | Company 2 | 5, tagged by our switch | vlan3 | 10.48.3.254/24 | Wifi for visitors | 3, over eth0 | vlan10 | 10.48.10.254/24 | Telephony | 10, over eth0 | vlan50 | 213.X.X.X/30 | ISP 3 | 50, over eth0 | vlan51 | 10.48.51.2/24 | ISP 2 | 51, over eth0 MTU is 1500 for eth1 ans 1492 for all others. eth1 has 2 public IPs, one of them is dedicated for VoIP and is "redirected" to our Asterisk server. From my previous message, here is what "ethtool -k" returns for eth1:> # ethtool -k eth1 > Offload parameters for eth1: > rx-checksumming: on > tx-checksumming: on > scatter-gather: on > tcp-segmentation-offload: off > udp-fragmentation-offload: off > generic-segmentation-offload: off > generic-receive-offload: off > large-receive-offload: off > ntuple-filters: off > receive-hashing: offFor all other interfaces, the only difference is:> tcp-segmentation-offload: on > generic-segmentation-offload: onMaybe the problem comes from our setup... I removed the IN-BANDWIDTH in tcdevides to "-". ============#NUMBER: IN-BANDWITH OUT-BANDWIDTH OPTIONS REDIRECTED #INTERFACE INTERFACES 1:eth1 - 10240kbit ============ Then no more download problem and QoS seems to work (limited output for some services). But is it a good workaround? I will upgrade the server to Debian Wheezy in January which provides Shorewall 4.5.5.3. Do you think it could solve the problem? Thanks, Olivier ----- Mail original ----- De: "Simon Matter" <simon.matter@invoca.ch> À: "Shorewall Users" <shorewall-users@lists.sourceforge.net> Envoyé: Jeudi 5 Décembre 2013 07:59:10 Objet: Re: [Shorewall-users] Multiple ISP + traffic shapping = poor download speed> It's not. > > # ethtool -k eth1 > Offload parameters for eth1: > rx-checksumming: on > tx-checksumming: on > scatter-gather: on > tcp-segmentation-offload: off > udp-fragmentation-offload: off > generic-segmentation-offload: off > generic-receive-offload: off > large-receive-offload: off > ntuple-filters: off > receive-hashing: offAnd what's on eth0 and eth2? What I don't understand is your interfaces file, where you have isp1 on eth0 and isp2 on eth1, while in another place you have isp1 -> eth1 and isp2 -> eth2. Is this all correct? Simon> > -Olivier > ----- Mail original ----- > De: "Tom Eastep" <teastep@shorewall.net> > À: "Shorewall Users" <shorewall-users@lists.sourceforge.net> > Envoyé: Mercredi 4 Décembre 2013 00:56:39 > Objet: Re: [Shorewall-users] Multiple ISP + traffic shapping > poor download speed > > On 12/3/2013 2:03 PM, olivier.monaco@free.fr wrote: >> Hello, >> >> Thanks for the great Shorewall which has replaced my hard to maintain >> home-made scripts. >> >> First, what works. >> >> Our local network is 10.48.X.X with multiple vlan, each on a dedicated >> interface. We use Shorewall 4.4.11 from Debian Squeeze. >> >> We have a 2 ISP: >> - isp1 : an optical fiber provider with 10 Mbps. >> - isp2 : a DSL provider with 15Mbits/1Mbits. >> >> We use isp2 as the default outgoing provider. The isp1 provider is used >> for "critical" services (SSH...) and for incoming connections (VPN...). >> >> Our interfaces file : >> =======================>> isp1 eth0 detect >> logmartians,nosmurfs,routefilter=0,tcpflags >> isp2 eth1 detect >> logmartians,nosmurfs,routefilter,tcpflags >> =======================>> >> Here is our providers file: >> =======================>> #NAME NUMBER MARK DUPLICATE INTERFACE GATEWAY >> OPTIONS COPY >> isp1 1 0x100 - eth1 37.X.X.X >> track,loose - >> isp2 2 0x200 - eth2 217.X.X.X >> track,balance - >> =======================>> >> Here is an extract of our tcrules file: >> =======================>> ###################################################################################################################### >> #MARK SOURCE DEST PROTO DEST SOURCE USER >> TEST LENGTH TOS CONNBYTES HELPER >> # PORT(S) PORT(S) >> >> # ISP1 DNS => ISP1 >> 256 0.0.0.0/0 37.X.X.X >> 256 $FW 37.X.X.X >> >> # ISP2 DNS => ISP2 >> 512 0.0.0.0/0 127.X.X.X >> 512 $FW 127.X.X.X >> >> # Google DNS => ISP1 >> 256 0.0.0.0/0 8.8.8.8,8.8.4.4 >> 256 $FW 8.8.8.8,8.8.4.4 >> >> # VPN IPsec (out) => ISP1 >> 256 0.0.0.0/0 0.0.0.0/0 udp 500,4500 >> 256 $FW 0.0.0.0/0 udp 500,4500 >> >> # Force one host to ISP1 >> 256 10.48.1.10 0.0.0.0/0 >> >> # Force all SSH to ISP1 >> 256 0.0.0.0/0 0.0.0.0/0 tcp 22 >> 256 $FW 0.0.0.0/0 tcp 22 >> =======================>> >> Yesterday we added VoIP. To do so, we force traffic from our Asterisk >> server to go throw ISP1 with a dedicated public IP and force the traffic >> from this dedicated public IP to go to Asterisk server (with IP >> filtering for security). This works too. >> >> Now, my problem is to put QoS (using TC_ENABLED=Internal). I try many >> configuration but always have the same problem: once the isp1 interface >> is listed in tcdevices, we have poor download speed. Even with/without >> other TC configuration. >> >> Here is our tcdevices file: >> =======================>> #NUMBER: IN-BANDWITH OUT-BANDWIDTH OPTIONS REDIRECTED >> #INTERFACE INTERFACES >> 1:isp1 10240kbit 10240kbit >> =======================>> >> We use an external server to test download speed with IP 5.X.X.X so we >> added in tcrules: >> =======================>> 256 0.0.0.0/0 5.X.X.X >> $FW 0.0.0.0/0 5.X.X.X >> =======================>> >> The results are: >> - without isp1 in tcdevices => more than 1MB/s (bytes measured with wget >> command) >> - with isp1 in tcdevices => less than 300 kB/s >> >> If I change bandwidth of isp1 to something more than 70000kbit, all goes >> right... Other lower value have the same problem but with different >> download speed (seems proportional to the interface speed). >> >> Here is a result of the following command: tc -s -d class show dev isp1 >> ======================== class htb 1:1 root rate 10240Kbit ceil >> 10240Kbit burst 1598b/8 mpu 0b overhead 0b cburst 1598b/8 mpu 0b >> overhead 0b level 7 >> Sent 1111091 bytes 11680 pkt (dropped 0, overlimits 0 requeues 0) >> rate 83656bit 124pps backlog 0b 0p requeues 0 >> lended: 0 borrowed: 0 giants: 0 >> tokens: 17781 ctokens: 17781 >> =======================>> >> Rates seems to be OK. >> >> Have someone the same problem? >> > > Sounds like Shorewall FAQ 97a. > > -Tom > -- > Tom Eastep \ When I die, I want to go like my Grandfather who > Shoreline, \ died peacefully in his sleep. Not screaming like > Washington, USA \ all of the passengers in his car > http://shorewall.net \________________________________________________ > > > ------------------------------------------------------------------------------ > Sponsored by Intel(R) XDK > Develop, test and display web and hybrid apps with a single code base. > Download it for free now! > http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk > _______________________________________________ > Shorewall-users mailing list > Shorewall-users@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/shorewall-users >------------------------------------------------------------------------------ Sponsored by Intel(R) XDK Develop, test and display web and hybrid apps with a single code base. Download it for free now! http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk _______________________________________________ Shorewall-users mailing list Shorewall-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/shorewall-users
Tom Eastep
2013-Dec-05 21:29 UTC
Re: Multiple ISP + traffic shapping = poor download speed
On 12/5/2013 11:55 AM, olivier.monaco@free.fr wrote:> Maybe the problem comes from our setup... > > I removed the IN-BANDWIDTH in tcdevides to "-". > ============> #NUMBER: IN-BANDWITH OUT-BANDWIDTH OPTIONS REDIRECTED > #INTERFACE INTERFACES > 1:eth1 - 10240kbit > ============> > Then no more download problem and QoS seems to work (limited output for some services). But is it a good workaround? > > I will upgrade the server to Debian Wheezy in January which provides Shorewall 4.5.5.3. Do you think it could solve the problem? >Possibly -- then you can configure a rate-estimating filter which provide much better accuracy than is available in 4.4.11. -Tom -- Tom Eastep \ When I die, I want to go like my Grandfather who Shoreline, \ died peacefully in his sleep. Not screaming like Washington, USA \ all of the passengers in his car http://shorewall.net \________________________________________________ ------------------------------------------------------------------------------ Sponsored by Intel(R) XDK Develop, test and display web and hybrid apps with a single code base. Download it for free now! http://pubads.g.doubleclick.net/gampad/clk?id=111408631&iu=/4140/ostg.clktrk