thr3ads.net - LARTC - Strange HTB behavior causing slow downloads [Mar 2004]

If this information is useful, please help other people find it:
Share via:

Dmitry

2004-Mar-05 09:12 UTC

Strange HTB behavior causing slow downloads

Hello ladies and gentlemen. I hope you might be able to help me with my
very strange problem.

I have a very typical setup - a LAN masquerading behind a Linux box.
This linux box has HTB going. It works surprisingly well for outbound
traffic (low latency stuff is low, pings are low, everything is great)
but there''s some strange behavior with downloads. Let me explain.

I have a bunch of streams going at a high port. Lets say their combined
downstream bandwidth is 20kbyte/s. When I remove the QoS, the bandwidth
shoots up considerably - to about 80kbyte/s. If I leave QoS on, but only
enable _one_ stream, the downstream bandwidth is similarly increased.

I do not have any sort of inbound bandwidth shaping. I''ve also tried to
prioritize ACKs above everything else. That didn''t seem to change
anything. The multiple-stream downloads are slow regardless of whether
the upstream bandwidth is fully utilized. The class that contains the
ACKs is never being used close to capacity. I''m completely stumped.

Hopefully you guys can spot something strange in the script below.
The service in question runs on ports 47504-47654.

Thanks in advance.

----------------------------------------------------------------
# http://lartc.org/howto/lartc.cookbook.fullnat.intro.html
export CEIL=374

# initial setup
tc qdisc add dev eth0 root handle 1: htb default 15
tc class add dev eth0 parent 1: classid 1:1 htb rate ${CEIL}kbit ceil
${CEIL}kbit
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 128kbit ceil ${CEIL}kbit
prio 0
tc class add dev eth0 parent 1:1 classid 1:11 htb rate 120kbit ceil ${CEIL}kbit
prio 1
tc class add dev eth0 parent 1:1 classid 1:12 htb rate 20kbit ceil ${CEIL}kbit
prio 2
tc class add dev eth0 parent 1:1 classid 1:13 htb rate 20kbit ceil ${CEIL}kbit
prio 2
tc class add dev eth0 parent 1:1 classid 1:14 htb rate 10kbit ceil ${CEIL}kbit
prio 3
tc class add dev eth0 parent 1:1 classid 1:15 htb rate 52kbit ceil ${CEIL}kbit
prio 3
tc class add dev eth0 parent 1:1 classid 1:16 htb rate 24kbit ceil ${CEIL}kbit
prio 4

tc qdisc add dev eth0 parent 1:12 handle 120: sfq perturb 10
tc qdisc add dev eth0 parent 1:13 handle 130: sfq perturb 10
tc qdisc add dev eth0 parent 1:14 handle 140: sfq perturb 10
tc qdisc add dev eth0 parent 1:15 handle 150: sfq perturb 10
tc qdisc add dev eth0 parent 1:16 handle 160: sfq perturb 10

# classify packets
tc filter add dev eth0 parent 1:0 protocol ip prio 1 handle 1 fw classid 1:10
tc filter add dev eth0 parent 1:0 protocol ip prio 2 handle 2 fw classid 1:11
tc filter add dev eth0 parent 1:0 protocol ip prio 3 handle 3 fw classid 1:12
tc filter add dev eth0 parent 1:0 protocol ip prio 4 handle 4 fw classid 1:13
tc filter add dev eth0 parent 1:0 protocol ip prio 5 handle 5 fw classid 1:14
tc filter add dev eth0 parent 1:0 protocol ip prio 6 handle 6 fw classid 1:15
tc filter add dev eth0 parent 1:0 protocol ip prio 7 handle 7 fw classid 1:16

# mark packets
for CHAIN in PREROUTING OUTPUT; do

	# prioritize small ACK packets above all else
	iptables -t mangle -A $CHAIN -p tcp --tcp-flags ALL ACK -m state --state
ESTABLISHED -m length --length :100 -j MARK --set-mark 0x1
	iptables -t mangle -A $CHAIN -p tcp --tcp-flags ALL ACK -m state --state
ESTABLISHED -m length --length :100 -j RETURN

	# prioritize small initial  domain/WWW requests for faster browsing
	iptables -t mangle -A $CHAIN -p tcp --dport www --syn -m state --state NEW -m
length --length 40:68 -j MARK --set-mark 0x1
	iptables -t mangle -A $CHAIN -p tcp --dport www --syn -m state --state NEW -m
length --length 40:68 -j RETURN
	iptables -t mangle -A $CHAIN -p tcp --dport domain --syn -m state --state NEW
-m length --length 40:68 -j MARK --set-mark 0x1
	iptables -t mangle -A $CHAIN -p tcp --dport domain --syn -m state --state NEW
-m length --length 40:68 -j RETURN

	# icmp gets priority
	iptables -t mangle -A $CHAIN -p icmp -j MARK --set-mark 0x1
	iptables -t mangle -A $CHAIN -p icmp -j RETURN


	# new connections packets get priority
	iptables -t mangle -A $CHAIN -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j MARK
--set-mark 0x1
	iptables -t mangle -A $CHAIN -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j
RETURN

	# prioritize domain requests
	iptables -t mangle -A $CHAIN  -p tcp --dport domain -j MARK --set-mark 0x2
	iptables -t mangle -A $CHAIN  -p tcp --dport domain -j RETURN
	iptables -t mangle -A $CHAIN  -p udp --dport domain -j MARK --set-mark 0x2
	iptables -t mangle -A $CHAIN  -p udp --dport domain -j RETURN

	# prioritize webserver requests
	iptables -t mangle -A $CHAIN  -p tcp --dport www -j MARK --set-mark 0x2
	iptables -t mangle -A $CHAIN  -p tcp --dport www -j RETURN

	iptables -t mangle -A $CHAIN  -p udp --dport www -j MARK --set-mark 0x2
	iptables -t mangle -A $CHAIN  -p udp --dport www -j RETURN


	# bulk - Torrents
	iptables -t mangle -A $CHAIN -p tcp --sport 6880:6899 -j MARK --set-mark 0x7
	iptables -t mangle -A $CHAIN -p tcp --sport 6880:6899 -j RETURN
	iptables -t mangle -A $CHAIN -p tcp --dport 6880:6899 -j MARK --set-mark 0x7
	iptables -t mangle -A $CHAIN -p tcp --dport 6880:6899 -j RETURN

	iptables -t mangle -A $CHAIN -p tcp --sport 47504:47654 -j MARK --set-mark 0x7
	iptables -t mangle -A $CHAIN -p tcp --sport 47504:47654 -j RETURN

	# edonkey
	iptables -t mangle -A $CHAIN -p tcp --dport 4662:4665 -j MARK --set-mark 0x7
	iptables -t mangle -A $CHAIN -p tcp --dport 4662:4665 -j RETURN

	# make ssh sorta interactive
	iptables -t mangle -A $CHAIN -p tcp -m tcp --sport 22 -j MARK --set-mark 0x2
	iptables -t mangle -A $CHAIN -p tcp -m tcp --sport 22 -j RETURN



	# funky TOS flags
	iptables -t mangle -A $CHAIN -m tos --tos Minimize-Delay -j MARK --set-mark 0x1
	iptables -t mangle -A $CHAIN -m tos --tos Minimize-Delay -j RETURN
	iptables -t mangle -A $CHAIN -m tos --tos Minimize-Cost -j MARK --set-mark 0x5
	iptables -t mangle -A $CHAIN -m tos --tos Minimize-Cost -j RETURN
	iptables -t mangle -A $CHAIN -m tos --tos Maximize-Throughput -j MARK
--set-mark 0x6
	iptables -t mangle -A $CHAIN -m tos --tos Maximize-Throughput -j RETURN


	# redundant.
	iptables -t mangle -A $CHAIN -j MARK --set-mark 0x6
done


_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

Dmitry

2004-Mar-05 09:22 UTC

head link

Strange HTB behavior causing slow downloads

Hello ladies and gentlemen. I hope you might be able to help me with my
very strange problem.

I have a very typical setup - a LAN masquerading behind a Linux box.
This linux box has HTB going. It works surprisingly well for outbound
traffic (low latency stuff is low, pings are low, everything is great)
but there''s some strange behavior with downloads. Let me explain.

I have a bunch of streams going at a high port. Lets say their combined
downstream bandwidth is 20kbyte/s. When I remove the QoS, the bandwidth
shoots up considerably - to about 80kbyte/s. If I leave QoS on, but only
enable _one_ stream, the downstream bandwidth is similarly increased.

I do not have any sort of inbound bandwidth shaping. I''ve also tried to
prioritize ACKs above everything else. That didn''t seem to change
anything. The multiple-stream downloads are slow regardless of whether
the upstream bandwidth is fully utilized. The class that contains the
ACKs is never being used close to capacity. I''m completely stumped.

Hopefully you guys can spot something strange in the script below.
The service in question runs on ports 47504-47654.

Thanks in advance.

----------------------------------------------------------------
# http://lartc.org/howto/lartc.cookbook.fullnat.intro.html
export CEIL=374

# initial setup
tc qdisc add dev eth0 root handle 1: htb default 15
tc class add dev eth0 parent 1: classid 1:1 htb rate ${CEIL}kbit ceil
${CEIL}kbit
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 128kbit ceil ${CEIL}kbit
prio 0
tc class add dev eth0 parent 1:1 classid 1:11 htb rate 120kbit ceil ${CEIL}kbit
prio 1
tc class add dev eth0 parent 1:1 classid 1:12 htb rate 20kbit ceil ${CEIL}kbit
prio 2
tc class add dev eth0 parent 1:1 classid 1:13 htb rate 20kbit ceil ${CEIL}kbit
prio 2
tc class add dev eth0 parent 1:1 classid 1:14 htb rate 10kbit ceil ${CEIL}kbit
prio 3
tc class add dev eth0 parent 1:1 classid 1:15 htb rate 52kbit ceil ${CEIL}kbit
prio 3
tc class add dev eth0 parent 1:1 classid 1:16 htb rate 24kbit ceil ${CEIL}kbit
prio 4

tc qdisc add dev eth0 parent 1:12 handle 120: sfq perturb 10
tc qdisc add dev eth0 parent 1:13 handle 130: sfq perturb 10
tc qdisc add dev eth0 parent 1:14 handle 140: sfq perturb 10
tc qdisc add dev eth0 parent 1:15 handle 150: sfq perturb 10
tc qdisc add dev eth0 parent 1:16 handle 160: sfq perturb 10

# classify packets
tc filter add dev eth0 parent 1:0 protocol ip prio 1 handle 1 fw classid 1:10
tc filter add dev eth0 parent 1:0 protocol ip prio 2 handle 2 fw classid 1:11
tc filter add dev eth0 parent 1:0 protocol ip prio 3 handle 3 fw classid 1:12
tc filter add dev eth0 parent 1:0 protocol ip prio 4 handle 4 fw classid 1:13
tc filter add dev eth0 parent 1:0 protocol ip prio 5 handle 5 fw classid 1:14
tc filter add dev eth0 parent 1:0 protocol ip prio 6 handle 6 fw classid 1:15
tc filter add dev eth0 parent 1:0 protocol ip prio 7 handle 7 fw classid 1:16

# mark packets
for CHAIN in PREROUTING OUTPUT; do

	# prioritize small ACK packets above all else
	iptables -t mangle -A $CHAIN -p tcp --tcp-flags ALL ACK -m state --state
ESTABLISHED -m length --length :100 -j MARK --set-mark 0x1
	iptables -t mangle -A $CHAIN -p tcp --tcp-flags ALL ACK -m state --state
ESTABLISHED -m length --length :100 -j RETURN

	# prioritize small initial  domain/WWW requests for faster browsing
	iptables -t mangle -A $CHAIN -p tcp --dport www --syn -m state --state NEW -m
length --length 40:68 -j MARK --set-mark 0x1
	iptables -t mangle -A $CHAIN -p tcp --dport www --syn -m state --state NEW -m
length --length 40:68 -j RETURN
	iptables -t mangle -A $CHAIN -p tcp --dport domain --syn -m state --state NEW
-m length --length 40:68 -j MARK --set-mark 0x1
	iptables -t mangle -A $CHAIN -p tcp --dport domain --syn -m state --state NEW
-m length --length 40:68 -j RETURN

	# icmp gets priority
	iptables -t mangle -A $CHAIN -p icmp -j MARK --set-mark 0x1
	iptables -t mangle -A $CHAIN -p icmp -j RETURN


	# new connections packets get priority
	iptables -t mangle -A $CHAIN -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j MARK
--set-mark 0x1
	iptables -t mangle -A $CHAIN -p tcp -m tcp --tcp-flags SYN,RST,ACK SYN -j
RETURN

	# prioritize domain requests
	iptables -t mangle -A $CHAIN  -p tcp --dport domain -j MARK --set-mark 0x2
	iptables -t mangle -A $CHAIN  -p tcp --dport domain -j RETURN
	iptables -t mangle -A $CHAIN  -p udp --dport domain -j MARK --set-mark 0x2
	iptables -t mangle -A $CHAIN  -p udp --dport domain -j RETURN

	# prioritize webserver requests
	iptables -t mangle -A $CHAIN  -p tcp --dport www -j MARK --set-mark 0x2
	iptables -t mangle -A $CHAIN  -p tcp --dport www -j RETURN

	iptables -t mangle -A $CHAIN  -p udp --dport www -j MARK --set-mark 0x2
	iptables -t mangle -A $CHAIN  -p udp --dport www -j RETURN


	# bulk - Torrents
	iptables -t mangle -A $CHAIN -p tcp --sport 6880:6899 -j MARK --set-mark 0x7
	iptables -t mangle -A $CHAIN -p tcp --sport 6880:6899 -j RETURN
	iptables -t mangle -A $CHAIN -p tcp --dport 6880:6899 -j MARK --set-mark 0x7
	iptables -t mangle -A $CHAIN -p tcp --dport 6880:6899 -j RETURN

	iptables -t mangle -A $CHAIN -p tcp --sport 47504:47654 -j MARK --set-mark 0x7
	iptables -t mangle -A $CHAIN -p tcp --sport 47504:47654 -j RETURN

	# edonkey
	iptables -t mangle -A $CHAIN -p tcp --dport 4662:4665 -j MARK --set-mark 0x7
	iptables -t mangle -A $CHAIN -p tcp --dport 4662:4665 -j RETURN

	# make ssh sorta interactive
	iptables -t mangle -A $CHAIN -p tcp -m tcp --sport 22 -j MARK --set-mark 0x2
	iptables -t mangle -A $CHAIN -p tcp -m tcp --sport 22 -j RETURN



	# funky TOS flags
	iptables -t mangle -A $CHAIN -m tos --tos Minimize-Delay -j MARK --set-mark 0x1
	iptables -t mangle -A $CHAIN -m tos --tos Minimize-Delay -j RETURN
	iptables -t mangle -A $CHAIN -m tos --tos Minimize-Cost -j MARK --set-mark 0x5
	iptables -t mangle -A $CHAIN -m tos --tos Minimize-Cost -j RETURN
	iptables -t mangle -A $CHAIN -m tos --tos Maximize-Throughput -j MARK
--set-mark 0x6
	iptables -t mangle -A $CHAIN -m tos --tos Maximize-Throughput -j RETURN


	# redundant.
	iptables -t mangle -A $CHAIN -j MARK --set-mark 0x6
done



_______________________________________________
LARTC mailing list / LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/

LARTC - Mar 2004 - Strange HTB behavior causing slow downloads

Strange HTB behavior causing slow downloads

Strange HTB behavior causing slow downloads