Hello - What I want to do seems very simple - I want to make sure any H.323 traffic gets processed before anything else entering or leaving this network. The network has a videoconferencing device on the LAN at 192.168.16.4. A Linux firewall NATs an external IP Address to this internal address and I have appropriate SNAT and DNAT rules that work. The NAT and connection tracking rules all work great. Now I need to make sure other traffic in and out of this network does not interfere with the H.323 flow. After pouring over several RFCs, Howto documents, and lots of other documentation, I think this very simple script will do the trick. The theory - by default, all interfaces have a classless PFIFO queue with three bands. So all I need to do is set the appropriate DS bits in the packet header to the EF (Expedited Forward) value and everything else will just work. Linux will put the packets in the top PFIFO priority band and they''ll go thru my Firewall at Warp 9.9 regardless of other traffic from other users. Assumptions: (1) I don''t care about slowing down other traffic flows. H.323 packets should be serviced first no matter what. (2) Any traffic with source or destination public IP Address "abc.def.ghi.jkl" or private IP Address 192.168.16.4 is to/from from the videoconference device. After all the reading and studying, is it really this simple? Does anyone have ideas on how to test this? How do I watch packets to see which packets go into what PFIFO band? #!/bin/sh VTC1_PRIVATE="192.168.16.4" VTC1_PUBLIC="abc.def.ghi.jkl" IPTABLES="/usr/local/sbin/iptables" $IPTABLES -t mangle -F $IPTABLES -t mangle -A PREROUTING -s $VTC1_PRIVATE -j DSCP --set-dscp-class EF $IPTABLES -t mangle -A PREROUTING -d $VTC1_PUBLIC -j DSCP --set-dscp-class EF thanks - Greg Scott
Maybe my idea from the other day isn''t so simple. When I run tcpdump and watch my h.323 packet stream, tcpdump shows a stream that looks like this: . . . 09:16:09.031943 abc.def.ghi.jkl.3230 > mno.pqr.stu.vwx.3230: udp 92 [tos 0xb8] 09:16:09.048128 mno.pqr.stu.vwx.3230 > abc.def.ghi.jkl.3230: udp 92 09:16:09.071137 abc.def.ghi.jkl.3232 > mno.pqr.stu.vwx.3232: udp 245 [tos 0xb8] 09:16:09.071535 abc.def.ghi.jkl.3232 > mno.pqr.stu.vwx.3232: udp 245 [tos 0xb8] 09:16:09.076762 mno.pqr.stu.vwx.3232 > abc.def.ghi.jkl.3232: udp 262 . . . where abc.dev... is the IP Addr of the listening H.323 device and mno.pqr... is the IP Addr of the calling system. This spawns 2 questions. First, what''s up with the tos bits? The TOS field is only 5 bits and it''s obsolete anyway, so clearly when tcpdump shows tos bits it must mean something else. Is tcpdump showing me the whole DS octet? Or the leftmost 6 bits of the DS octet? Or what? From what I''ve been reading, that DS octet has been carved up so many different ways over the years that I am now hopelessly confused. Next, why don''t all packets show a tos field? My iptables rules should put a value in the DS octet for everything to/from that IP Address (see below). So why aren''t they in there? Here are some test results. In the test below, the conversation used 768 packets. Yet it only set the DSCP bits in 472 packets. And tcpdump shows a different value anyway. What''s going on here???? (Note that I modified the script from last night slightly as a troubleshooting step. That''s why there are 4 rules below instead of two rules.) [root@fw firewall-scripts]# /usr/sbin/tcpdump -i eth0 host abc.def.ghi.jkl -n . . . 768 packets received by filter 0 packets dropped by kernel [root@fw firewall-scripts]# /usr/local/sbin/iptables -L -v -n -t mangle Chain PREROUTING (policy ACCEPT 25M packets, 14G bytes) pkts bytes target prot opt in out source destination 293 64981 DSCP all -- * * 192.168.16.4 0.0.0.0/0 DSCP set 0x2e 0 0 DSCP all -- * * 0.0.0.0/0 192.168.16.4 DSCP set 0x2e 0 0 DSCP all -- * * abc.def.ghi.jkl 0.0.0.0/0 DSCP set 0x2e 179 36722 DSCP all -- * * 0.0.0.0/0 abc.def.ghi.jkl DSCP set 0x2e Here is the exact script that generated the rules above: [root@fw firewall-scripts]# more vtc-ds.sh #!/bin/sh VTC1_PRIVATE="192.168.16.4" VTC1_PUBLIC="abc.def.ghi.jkl" IPTABLES="/usr/local/sbin/iptables" $IPTABLES -t mangle -F $IPTABLES -t mangle -A PREROUTING -s $VTC1_PRIVATE -j DSCP --set-dscp-class EF $IPTABLES -t mangle -A PREROUTING -d $VTC1_PRIVATE -j DSCP --set-dscp-class EF $IPTABLES -t mangle -A PREROUTING -s $VTC1_PUBLIC -j DSCP --set-dscp-class EF $IPTABLES -t mangle -A PREROUTING -d $VTC1_PUBLIC -j DSCP --set-dscp-class EF [root@fw firewall-scripts]# - Greg Scott -----Original Message----- From: lartc-bounces@mailman.ds9a.nl [mailto:lartc-bounces@mailman.ds9a.nl] On Behalf Of Greg Scott Sent: Sunday, May 29, 2005 11:15 PM To: lartc@mailman.ds9a.nl Subject: [LARTC] Very simple traffic shaping script for H.323 Hello - What I want to do seems very simple - I want to make sure any H.323 traffic gets processed before anything else entering or leaving this network. The network has a videoconferencing device on the LAN at 192.168.16.4. A Linux firewall NATs an external IP Address to this internal address and I have appropriate SNAT and DNAT rules that work. The NAT and connection tracking rules all work great. Now I need to make sure other traffic in and out of this network does not interfere with the H.323 flow. After pouring over several RFCs, Howto documents, and lots of other documentation, I think this very simple script will do the trick. The theory - by default, all interfaces have a classless PFIFO queue with three bands. So all I need to do is set the appropriate DS bits in the packet header to the EF (Expedited Forward) value and everything else will just work. Linux will put the packets in the top PFIFO priority band and they''ll go thru my Firewall at Warp 9.9 regardless of other traffic from other users. Assumptions: (1) I don''t care about slowing down other traffic flows. H.323 packets should be serviced first no matter what. (2) Any traffic with source or destination public IP Address "abc.def.ghi.jkl" or private IP Address 192.168.16.4 is to/from from the videoconference device. After all the reading and studying, is it really this simple? Does anyone have ideas on how to test this? How do I watch packets to see which packets go into what PFIFO band? #!/bin/sh VTC1_PRIVATE="192.168.16.4" VTC1_PUBLIC="abc.def.ghi.jkl" IPTABLES="/usr/local/sbin/iptables" $IPTABLES -t mangle -F $IPTABLES -t mangle -A PREROUTING -s $VTC1_PRIVATE -j DSCP --set-dscp-class EF $IPTABLES -t mangle -A PREROUTING -d $VTC1_PUBLIC -j DSCP --set-dscp-class EF thanks - Greg Scott _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
I may have the answer to my own questions: (1) When the value for the DSCP field for EF (Expedited Flow) is hex 2E, why does tcpdump display it as hex B8? I think there''s some bit shifting going on. The DSCP field is the left most 6 bits of the DS octet in the packet header. 2E is 0010 1110. But really, we only care about the leftmost 6 bits, so it''s really 10 1110. I''ll bet tcpdump displays the whole octet, not just those 6 bits. The rightmost bits are 0. So add them on to the right and it''s 10 1110 00. Or, putting it together like hex digits, 1011 1000, or hex b8. And, sure enough, this is what tcpdump shows me. So it''s gotta be that tcpdump displays the whole DS octet, but it confuses me by calling it tos. (2) Assuming the tos value tcpdump shows me makes sense, why do some packets in the tcpdump output stream show tos values and others don''t? This just hit me. Examining the output more closely, the packets do NOT show tos values are inbound. The packets that DO show tos values are outbound. As I think about it, this kind of makes sense. TCPDUMP puts the NIC in promiscous mode and listens directly to it. So it grabs and displays inbound packets right off the NIC, before they ever get into the kernel. i.e. - tcpdump displays them before the kernel puts in those bits in the mangle table. Outbound is opposite. On the way out, the packets have gone through all the kernel processing and now they''re on their way out. So outbound packets will show the tos values because my firewall rules put them in. Cool! It also occurs to me, I can test this by looking at the internal interface, which moves packets to/from the internal LAN. If my theory is correct, and given the rules in my little script, then I should see the opposite behavior. Outbound packets should NOT show a tos value, and inbound from the Internet should show the tos value my firewall rules put in, shiftet by 2 bits. So trying my little experiment, the results are not exactly what I expected, but they still fit the theory: [root@fw firewall-scripts]# /usr/sbin/tcpdump -i eth1 host 192.168.16.4 -n tcpdump: listening on eth1 . . . 15:57:15.244738 mno.pqr.stu.vwx.3230 > 192.168.16.4.3230: udp 92 [tos 0xb8] 15:57:15.268208 192.168.16.4.3232 > mno.pqr.stu.vwx.3232: udp 219 [tos 0x80] 15:57:15.298114 192.168.16.4.3230 > mno.pqr.stu.vwx.3230: udp 92 [tos 0xa0] 15:57:15.310663 192.168.16.4.3231 > mno.pqr.stu.vwx.3231: udp 64 15:57:15.338424 192.168.16.4.3232 > mno.pqr.stu.vwx.3232: udp 427 [tos 0x80] 15:57:15.358067 192.168.16.4.3230 > mno.pqr.stu.vwx.3230: udp 92 [tos 0xa0] . . . 452 packets received by filter 0 packets dropped by kernel [root@fw firewall-scripts]# I think I know what''s going on. The near-end H.323 device has its own QOS settings and I''ll bet it''s putting these in when it asembles IP packets. H.323 has a bunch of separate sub-conversations, including audio, video, and remote camera control. When I look at the settings on the device, each of these has its own QOS value. The far-end device has all its QOS stuff turned off. So the internal interface has QOS stuff already set when each packet hits the NIC. Of course, the firewall replaces these with its own QOS values in the mangle table later on, but we don''t see that yet for outbound packets. If anyone has waded through all this, am I making sense? Also, now that I''m convinced I really am writing the DSCP values I want, how do I know whether the packets really go into that first priority PFIFO band? Thanks - Greg Scott -----Original Message----- From: lartc-bounces@mailman.ds9a.nl [mailto:lartc-bounces@mailman.ds9a.nl] On Behalf Of Greg Scott Sent: Monday, May 30, 2005 10:10 AM To: lartc@mailman.ds9a.nl Subject: RE: [LARTC] Very simple traffic shaping script for H.323 Maybe my idea from the other day isn''t so simple. When I run tcpdump and watch my h.323 packet stream, tcpdump shows a stream that looks like this: . . . 09:16:09.031943 abc.def.ghi.jkl.3230 > mno.pqr.stu.vwx.3230: udp 92 [tos 0xb8] 09:16:09.048128 mno.pqr.stu.vwx.3230 > abc.def.ghi.jkl.3230: udp 92 09:16:09.071137 abc.def.ghi.jkl.3232 > mno.pqr.stu.vwx.3232: udp 245 [tos 0xb8] 09:16:09.071535 abc.def.ghi.jkl.3232 > mno.pqr.stu.vwx.3232: udp 245 [tos 0xb8] 09:16:09.076762 mno.pqr.stu.vwx.3232 > abc.def.ghi.jkl.3232: udp 262 . . . where abc.dev... is the IP Addr of the listening H.323 device and mno.pqr... is the IP Addr of the calling system. This spawns 2 questions. First, what''s up with the tos bits? The TOS field is only 5 bits and it''s obsolete anyway, so clearly when tcpdump shows tos bits it must mean something else. Is tcpdump showing me the whole DS octet? Or the leftmost 6 bits of the DS octet? Or what? From what I''ve been reading, that DS octet has been carved up so many different ways over the years that I am now hopelessly confused. Next, why don''t all packets show a tos field? My iptables rules should put a value in the DS octet for everything to/from that IP Address (see below). So why aren''t they in there? Here are some test results. In the test below, the conversation used 768 packets. Yet it only set the DSCP bits in 472 packets. And tcpdump shows a different value anyway. What''s going on here???? (Note that I modified the script from last night slightly as a troubleshooting step. That''s why there are 4 rules below instead of two rules.) [root@fw firewall-scripts]# /usr/sbin/tcpdump -i eth0 host abc.def.ghi.jkl -n . . . 768 packets received by filter 0 packets dropped by kernel [root@fw firewall-scripts]# /usr/local/sbin/iptables -L -v -n -t mangle Chain PREROUTING (policy ACCEPT 25M packets, 14G bytes) pkts bytes target prot opt in out source destination 293 64981 DSCP all -- * * 192.168.16.4 0.0.0.0/0 DSCP set 0x2e 0 0 DSCP all -- * * 0.0.0.0/0 192.168.16.4 DSCP set 0x2e 0 0 DSCP all -- * * abc.def.ghi.jkl 0.0.0.0/0 DSCP set 0x2e 179 36722 DSCP all -- * * 0.0.0.0/0 abc.def.ghi.jkl DSCP set 0x2e Here is the exact script that generated the rules above: [root@fw firewall-scripts]# more vtc-ds.sh #!/bin/sh VTC1_PRIVATE="192.168.16.4" VTC1_PUBLIC="abc.def.ghi.jkl" IPTABLES="/usr/local/sbin/iptables" $IPTABLES -t mangle -F $IPTABLES -t mangle -A PREROUTING -s $VTC1_PRIVATE -j DSCP --set-dscp-class EF $IPTABLES -t mangle -A PREROUTING -d $VTC1_PRIVATE -j DSCP --set-dscp-class EF $IPTABLES -t mangle -A PREROUTING -s $VTC1_PUBLIC -j DSCP --set-dscp-class EF $IPTABLES -t mangle -A PREROUTING -d $VTC1_PUBLIC -j DSCP --set-dscp-class EF [root@fw firewall-scripts]# - Greg Scott -----Original Message----- From: lartc-bounces@mailman.ds9a.nl [mailto:lartc-bounces@mailman.ds9a.nl] On Behalf Of Greg Scott Sent: Sunday, May 29, 2005 11:15 PM To: lartc@mailman.ds9a.nl Subject: [LARTC] Very simple traffic shaping script for H.323 Hello - What I want to do seems very simple - I want to make sure any H.323 traffic gets processed before anything else entering or leaving this network. The network has a videoconferencing device on the LAN at 192.168.16.4. A Linux firewall NATs an external IP Address to this internal address and I have appropriate SNAT and DNAT rules that work. The NAT and connection tracking rules all work great. Now I need to make sure other traffic in and out of this network does not interfere with the H.323 flow. After pouring over several RFCs, Howto documents, and lots of other documentation, I think this very simple script will do the trick. The theory - by default, all interfaces have a classless PFIFO queue with three bands. So all I need to do is set the appropriate DS bits in the packet header to the EF (Expedited Forward) value and everything else will just work. Linux will put the packets in the top PFIFO priority band and they''ll go thru my Firewall at Warp 9.9 regardless of other traffic from other users. Assumptions: (1) I don''t care about slowing down other traffic flows. H.323 packets should be serviced first no matter what. (2) Any traffic with source or destination public IP Address "abc.def.ghi.jkl" or private IP Address 192.168.16.4 is to/from from the videoconference device. After all the reading and studying, is it really this simple? Does anyone have ideas on how to test this? How do I watch packets to see which packets go into what PFIFO band? #!/bin/sh VTC1_PRIVATE="192.168.16.4" VTC1_PUBLIC="abc.def.ghi.jkl" IPTABLES="/usr/local/sbin/iptables" $IPTABLES -t mangle -F $IPTABLES -t mangle -A PREROUTING -s $VTC1_PRIVATE -j DSCP --set-dscp-class EF $IPTABLES -t mangle -A PREROUTING -d $VTC1_PUBLIC -j DSCP --set-dscp-class EF thanks - Greg Scott _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc _______________________________________________ LARTC mailing list LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Greg Scott wrote:> The theory - by default, all interfaces have a classless PFIFO queue > with three bands. So all I need to do is set the appropriate DS bits in > the packet header to the EF (Expedited Forward) value and everything > else will just work. Linux will put the packets in the top PFIFO > priority band and they''ll go thru my Firewall at Warp 9.9 regardless of > other traffic from other users. > > Assumptions: > (1) I don''t care about slowing down other traffic flows. H.323 packets > should be serviced first no matter what. > (2) Any traffic with source or destination public IP Address > "abc.def.ghi.jkl" or private IP Address 192.168.16.4 is to/from from the > videoconference device.I can''t really explain all what you see playing with dsmark - but you may have to do things differently anyway. The pfifo fasts that default on interfaces are not going to help you if the link you are shaping has less bandwidth than your eths are capable of. I assume you want to shape traffic that is going over the internet? Andy.
Yup, Andy is right. This is traffic to/from the Internet. After brooding all day about this, my simple scheme to set TOS bits and let nature take its course had major holes: (1) Other apps could set the same TOS bits to any value they want, and those apps would slog it out with my H.323 stuff. Some apps, such as telnet, ssh, and others behave this way by default. This is not good. (2) I could not come up with any way to see what bands the packets were really going to anyway. So I could have the best settings in the world, but with no way to monitor them, they are useless. I needed a way to really control which packets go to what queues and I need a mechanism to monitor the results. So I threw everything out and started over again. Here''s the latest script: Thanks to Bert H for that howto. I lifted code directly from chapter 9 and I''m standing on the shoulders of a giant. I set up PRIO qdiscs on each interface and even waded into some details around the u32 filter. Each PRIO qdisc has 3 bands. I set up PFIFO qdiscs for each band. The idea is, traffic will flow with its default behavior within each band. So traffic to/from the VTC device will hopefully honor the TOS bits inside its band, and lower priority traffic will also sort itself out within its band. If I''m reading all the documentation correctly, that''s what PFIFO is supposed to do, right? I got really gutsy and used U32 for my filter. I may change this to use FWMARK later on. I read that U32 is the fastest filter - and this is good. But I couldn''t come up with a good way to directly monitor which packets hit what filter. But if I use FWMARK, I can log packets for debuggig purposes, and I track exactly which packets trigger what filter if I want. This would be extremely useful for debugging. With U32, the best I can do is watch the end result by looking at qdisc statistics. But I haven''t found a direct way to tell which filter sent what packets to which band. Well, here is the script. If anyone else is trying to shape H.323, I would love to hear your ideas. [root@fw firewall-scripts]# more priotest3.sh #!/bin/sh # Useful commands: # # $TC qdisc show Show the various qdiscs # $TC -s qdisc ls dev eth0 Show packet stats on eth0 # $TC filter show Show filters # $TC qdisc del dev eth0 root Set eth0 back to its defaults # # This script sets up 3 band PRIO qdiscs on both the external and # internal interfaces. Each band, in turn, has a PFIFO qdisc. # I chose PFIFO qdiscs in each band to preserve default behavior # within each band. Any traffic to/from the VTC device is # directed to the highest priority PRIO band. Hopefully, within # that band, the trafic follows default behavor and prioritizes # according to the TOS bits set by the application. # Everything else goes to the next priority band, # which should also follow default behavior within that # band. The idea is, we want the VTC traffic to get the highest # priority service, no matter what. Everything else should # follow default behavior, underneath the VTC flow. # # Greg Scott May 31, 2005 # IPTABLES="/usr/local/sbin/iptables" TC="/sbin/tc" INET_IFACE="eth0" TRUSTED1_IFACE="eth1" VTC1_PUBLIC="abc.def.ghi.jkl" VTC1_PRIVATE="192.168.16.4" $TC qdisc del dev $INET_IFACE root $TC qdisc del dev $TRUSTED1_IFACE root $TC qdisc add dev $INET_IFACE root handle 1: prio # This *instantly* creates classes 1:1, 1:2, 1:3 $TC qdisc add dev $TRUSTED1_IFACE root handle 2: prio # This *instantly* creates classes 2:1, 2:2, 2:3 $TC qdisc add dev $INET_IFACE parent 1:1 handle 11: pfifo $TC qdisc add dev $INET_IFACE parent 1:2 handle 12: pfifo $TC qdisc add dev $INET_IFACE parent 1:3 handle 13: pfifo $TC qdisc add dev $TRUSTED1_IFACE parent 2:1 handle 21: pfifo $TC qdisc add dev $TRUSTED1_IFACE parent 2:2 handle 22: pfifo $TC qdisc add dev $TRUSTED1_IFACE parent 2:3 handle 23: pfifo # # This assigns traffic to/from $VTC1_PUBLIC and $VTC1_PRIVATE # to the highest priority band of the queue for the appropriate # interface, and the rest to the next-highest proirity band. # $TC filter add dev $INET_IFACE parent 1:0 protocol ip prio 1 u32 \ match ip dst $VTC1_PUBLIC flowid 1:1 $TC filter add dev $INET_IFACE parent 1:0 protocol ip prio 1 u32 \ match ip src $VTC1_PUBLIC flowid 1:1 $TC filter add dev $TRUSTED1_IFACE parent 2:0 protocol ip prio 1 u32 \ match ip src $VTC1_PRIVATE flowid 2:1 $TC filter add dev $TRUSTED1_IFACE parent 2:0 protocol ip prio 1 u32 \ match ip dst $VTC1_PRIVATE flowid 2:1 $TC filter add dev $INET_IFACE parent 1:0 protocol ip prio 2 u32 \ match ip src 0.0.0.0/0 flowid 1:2 $TC filter add dev $TRUSTED1_IFACE parent 2:0 protocol ip prio 2 u32 \ match ip src 0.0.0.0/0 flowid 2:2 [root@fw firewall-scripts]# -----Original Message----- From: Andy Furniss [mailto:andy.furniss@dsl.pipex.com] Sent: Monday, May 30, 2005 5:59 PM To: Greg Scott Cc: lartc@mailman.ds9a.nl Subject: Re: [LARTC] Very simple traffic shaping script for H.323 Greg Scott wrote:> The theory - by default, all interfaces have a classless PFIFO queue > with three bands. So all I need to do is set the appropriate DS bits > in the packet header to the EF (Expedited Forward) value and > everything else will just work. Linux will put the packets in the top> PFIFO priority band and they''ll go thru my Firewall at Warp 9.9 > regardless of other traffic from other users. > > Assumptions: > (1) I don''t care about slowing down other traffic flows. H.323 > packets should be serviced first no matter what. > (2) Any traffic with source or destination public IP Address > "abc.def.ghi.jkl" or private IP Address 192.168.16.4 is to/from from > the videoconference device.I can''t really explain all what you see playing with dsmark - but you may have to do things differently anyway. The pfifo fasts that default on interfaces are not going to help you if the link you are shaping has less bandwidth than your eths are capable of. I assume you want to shape traffic that is going over the internet? Andy.