(list admin, please cancel the same post from my other email address -- forgot to change it on first submission) I need to setup QoS on a linux router/firewall I maintain. I spent 10 hours reading everything I could find on QoS/HTB/iproute2 and came up with what I thought made sense for my situation. So I deployed it and BOOM! KERNEL PANIC! Not what I was expecting... now the debugging begins. I reproduced the panic twice on two different (yet almost identically configured) machines. I can reproduce the panic on demand by doing a specific set of actions. First, my setup: I have 2 machines at different locations connected via internet. Both machines are stock Fedora Core 1 kernel 2.4.22-1.2179.nptl. I run free/SWAN (stock FC binary rpm's) between the 2 machines for ipsec VPN. I run VoIP, VNC and all other inter-office traffic through the VPN. The internet connection is ADSL with 400kbits/s up and 1500 or so down. VoIP is routed but not MASQ'd. VNC is MASQ'd (neither the originating nor destination machines are the linux boxes themseleves). Second, my goals: Give a fixed minimum bandwidth and high priority to VoIP through VPN. Same, but less so, for VNC through VPN. Give the VPN high enough allocation for VoIP and VNC to get through ok. Less important little tweaks for rarely-used outside (non IPSEC) VNC and ssh access. My situation seems different from the examples I've seen because *I believe* I need to have 2 completely separate qdiscs, 1 for ppp0 (the DSL) and 1 for ipsec0 (the freeSWAN VPN). Yet ipsec0 eventually goes over ppp0 so they are intertwined. I have a funny feeling this is where the crash is coming from. See my setup script near the bottom of this email (excuse the wrapping). Everything seemed to go great until I tried VNC'ing in from one office to the other. The VNC screen would pop up, do a first draw, then completely freeze. From that point on the remote linux router is frozen -- kernel panic. Strange that the bug would only trigger AFTER sending the 100-200kB of the initial VNC screen. Looking at my config, I will note a couple of questions I had while writing it that weren't answered in the docs I found: 1. The "tc filter add ... protocol ip" thing confused me. What exactly is the "protocol ip" for? I originally though that it should read "protocol 50" for the ipsec stuff, but that didn't seem to catch the packets, so I switched it back to "ip". Weird, while testing with it set to 50 (and having no packets match the rule) there were no crashes. 2. The iptables mangle rules will in the case of VNC and ssh *over VPN* match two rules. I *assume* the last executing MARK will overwrite the previous MARK. If for some reason the marks are ANDed or something, perhaps that is causing the crash (filtering 1 packet into 2 buckets?). 3. As I mentioned above, the fact that one qdisc will feed a separate qdisc, because ipsec0 eventually goes out over ppp0, may be a problem? I wish I had seen some examples of this type of setup. 4. I chose HTB instead of CBQ as it seemed simpler (always a good thing) and more suited to my exact needs. Not sure if the bug is in HTB itself or the general QoS stuff. my setup script: $iext=ppp0 $isec=ipsec0 $ivoi=eth3 $qosbw=380 # VNC iptables -t mangle -A PREROUTING -p tcp --sport 5900 -j MARK --set-mark 11 iptables -t mangle -A PREROUTING -p tcp --dport 5900 -j MARK --set-mark 11 iptables -t mangle -A PREROUTING -i $ivoi -j MARK --set-mark 10 iptables -t mangle -A OUTPUT -p 50 -j MARK --set-mark 10 iptables -t mangle -A OUTPUT -p 51 -j MARK --set-mark 10 iptables -t mangle -A OUTPUT -o $iext -p tcp --sport ssh -j MARK --set-mark 12 tc qdisc del dev $isec root >/dev/null 2>&1 tc qdisc add dev $isec root handle 1:0 htb default 13 tc class add dev $isec parent 1:0 classid 1:1 htb rate "$qosbw"kbit ceil "$qosbw"kbit tc class add dev $isec parent 1:1 classid 1:10 htb rate 160kbit ceil "$qosbw"kbit tc class add dev $isec parent 1:1 classid 1:11 htb rate 210kbit ceil "$qosbw"kbit tc class add dev $isec parent 1:1 classid 1:13 htb rate 010kbit ceil "$qosbw"kbit tc qdisc add dev $isec parent 1:10 handle 110:0 sfq perturb 10 tc qdisc add dev $isec parent 1:11 handle 111:0 sfq perturb 10 tc qdisc add dev $isec parent 1:13 handle 113:0 sfq perturb 10 tc filter add dev $isec parent 1:0 protocol ip handle 10 fw flowid 1:10 tc filter add dev $isec parent 1:0 protocol ip handle 11 fw flowid 1:11 tc qdisc del dev $iext root >/dev/null 2>&1 tc qdisc add dev $iext root handle 1:0 htb default 13 tc class add dev $iext parent 1:0 classid 1:1 htb rate "$qosbw"kbit ceil "$qosbw"kbit tc class add dev $iext parent 1:1 classid 1:10 htb rate 300kbit ceil "$qosbw"kbit tc class add dev $iext parent 1:1 classid 1:11 htb rate 050kbit ceil "$qosbw"kbit tc class add dev $iext parent 1:1 classid 1:12 htb rate 020kbit ceil "$qosbw"kbit tc class add dev $iext parent 1:1 classid 1:13 htb rate 010kbit ceil "$qosbw"kbit tc qdisc add dev $iext parent 1:10 handle 110:0 sfq perturb 10 tc qdisc add dev $iext parent 1:11 handle 111:0 sfq perturb 10 tc qdisc add dev $iext parent 1:12 handle 112:0 sfq perturb 10 tc qdisc add dev $iext parent 1:13 handle 113:0 sfq perturb 10 tc filter add dev $iext parent 1:0 protocol ip handle 10 fw flowid 1:10 tc filter add dev $iext parent 1:0 protocol ip handle 11 fw flowid 1:11 tc filter add dev $iext parent 1:0 protocol ip handle 12 fw flowid 1:12 The info dumped on-screen from the kernel panic. I couldn't find any way to scroll up and didn't have sysrq enabled and didn't have the ability to enable and reproduce (system was being used live in production during business hours!). I could potentially go back off-hours and reproduce with sysrq and get more info (hopefully). There may be slight typos as this was manually copied to paper and then back into this email! ... anything above not visible... eax 0 wbx 8 ecx 1 edx d741c001 esi c0384000 edi 0 ebp d741c009 esp c0385ef8 ds 68 es 68 ss 60 process (pid 0 stackpage c0385000) stack ddfc9000 ddfc9244 c0385f40 0 1 c0385f8c 0 c0125e93 c038400 0 1 0 c0385f8c 20000001 c0126278 0 c010ed90 c0385f8c c033ddb0 20000001 c010b1a5 0 0 c0385f8e call trace c0125e93 update_process_times [k] 0x33 0xc0385f14 c0126278 do_timer [k] 0x28 0xc0385f30 c010ed90 timer_interrupt [k] 0x80 0xc0385f38 c010b1a5 handle_IRQ_event [k] 0x45 0xc0385f48 c010b324 do_IRQ [k] 0x64 0xc0385f68 c010db28 call_do_IRQ [k] 0x05 0xc0385f88 c0110068 restore_i387 [k] 0x28 0xc0385fa8 c0106fb3 default_idle [k] 0x23 0xc0385fb4 c0115a7c apm_cpu_idle [k] 0xac 0xc0385fc0 c01159d0 apm_cpu_idle [k] 0x00 0xc0385fc4 c0107032 cpu_idle [k] 0x32 0xc0385fd4 c0105000 stext [k] 0x00 0xc0385fe0 code 01 b8 d0 01 00 00 01 88 d4 01 00 00 b8 1f 85 eb 51 89 96 c4 <0> kernel panic: Aiee, killing int handler! In interrupt handler not syncing Lastly, thanks!