Hi all,
I''m having a hell of a time getting my IFB to work. I know
I''ve done
this before, so I''m missing something stupid. Can anybody tell me what
it might be?
Configs as follows:
--------
#!/bin/sh
modprobe ifb numifbs=1000
modprobe act_mirred
modprobe 8021q
brctl addbr br0
brctl setfd br0 0
brctl stp br0 off
brctl addif br0 eth1
brctl addif br0 eth2
ifconfig eth1 up
ifconfig eth2 up
ifconfig br0 up
tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 100Mbit avpkt 1000
cell 8
tc qdisc add dev eth2 root handle 1:0 cbq bandwidth 100Mbit avpkt 1000
cell 8
ifconfig ifb0 up
tc qdisc add dev ifb0 root handle 1:0 cbq bandwidth 100Mbit avpkt 1000
cell 8
tc class add dev ifb0 parent 1:0 classid 1:1 cbq bandwidth 100Mbit rate
100Kbit weight 10Kbit prio 5 allot 1514 cell 8 maxburst 20 avpkt 1000
bounded isolated
tc qdisc add dev eth1 ingress
tc qdisc add dev eth2 ingress
tc filter add dev eth1 parent ffff: protocol ip prio 10 u32 match u32 0
0 action mirred egress redirect dev ifb0
tc filter add dev eth2 parent ffff: protocol ip prio 10 u32 match u32 0
0 action mirred egress redirect dev ifb0
tc filter add dev ifb0 parent 1:0 protocol ip prio 1 u32 match u32 0 0
flowid 1:1
--------------
By my reckoning, this should redirect all packets passing through eth2
and eth1 through the IFB, and put a rate limit on the IFB of
100Kbits/sec.
What have I missed?
Regards,
Leigh
Leigh Sharpe
Network Systems Engineer
Pacific Wireless
Ph +61 3 9584 8966
Mob 0408 009 502
Helpdesk 1300 300 616
email lsharpe@pacificwireless.com.au
web www.pacificwireless.com.au
_______________________________________________
LARTC mailing list
LARTC@mailman.ds9a.nl
http://mailman.ds9a.nl/cgi-bin/mailman/listinfo/lartc
Leigh Sharpe wrote:> Hi all, > I''m having a hell of a time getting my IFB to work. I know I''ve done > this before, so I''m missing something stupid. Can anybody tell me what > it might be? > Configs as follows: > > -------- > #!/bin/sh > modprobe ifb numifbs=1000 > modprobe act_mirred > modprobe 8021q > > brctl addbr br0 > brctl setfd br0 0 > brctl stp br0 off > brctl addif br0 eth1 > brctl addif br0 eth2 > ifconfig eth1 up > ifconfig eth2 up > ifconfig br0 up > > tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 100Mbit avpkt 1000 > cell 8 > tc qdisc add dev eth2 root handle 1:0 cbq bandwidth 100Mbit avpkt 1000 > cell 8 > > ifconfig ifb0 up > > tc qdisc add dev ifb0 root handle 1:0 cbq bandwidth 100Mbit avpkt 1000 > cell 8 > > tc class add dev ifb0 parent 1:0 classid 1:1 cbq bandwidth 100Mbit rate > 100Kbit weight 10Kbit prio 5 allot 1514 cell 8 maxburst 20 avpkt 1000 > bounded isolated > > tc qdisc add dev eth1 ingress > tc qdisc add dev eth2 ingress > tc filter add dev eth1 parent ffff: protocol ip prio 10 u32 match u32 0 > 0 action mirred egress redirect dev ifb0 > tc filter add dev eth2 parent ffff: protocol ip prio 10 u32 match u32 0 > 0 action mirred egress redirect dev ifb0 > > tc filter add dev ifb0 parent 1:0 protocol ip prio 1 u32 match u32 0 0 > flowid 1:1 > -------------- > > By my reckoning, this should redirect all packets passing through eth2 > and eth1 through the IFB, and put a rate limit on the IFB of > 100Kbits/sec. > What have I missed?Subject changed as the no subject thread I see is quite long and diverse :-) It will only get ingress ethertype ip traffic on eth 1/2 (I don''t know whether bridging complicates it - still havent fixed my own bridge yet) You need two more ... action mirred egress redirect ... to get egress eth1/2 traffic aswell. You need something classfull on the roots for it to work - you have CBQ already if you didn''t prio would have done. If you have a qdisc on the real interface and redirect egress traffic to ifb then you can''t classify again, though classification done before the redirect will be reinstated after the packets come back from ifb. Generally - I never used CBQ so don''t know it''s limitations/behavior. Accepting this may just be a test, but if you did it like this for real ie. sending in/out traffic to one slow buffer it will make for a truly horrible user experience - you should at least seperate the directions into seperate classes and make the classes share the bandwidth. I wonder if using policers and shared meters would be a nicer alternative - not that I have any examples to hand. If the nics are gig/ you use xen turn off * segmentation offload with ethtool -k Andy.
>Subject changed as the no subject thread I see is quite long and diverse :-)Yeah, sorry about that. Minor brain fade. Seems that the example I gave actually works, but not the way I''m using it. I am bridging VLAN tagged packets, but for some reason they are not being subjected to the rate limit. If I pass normal, untagged packets through this setup, it behaves as expected. However, once I put tagged packets through the bridge, it fails to shape traffic. I don''t want to have to use VLAN sub-interfaces, because the VLAN code strips the 802.1q tag from packets before they can be examined, which causes me problems in other areas. Regards, Leigh Leigh Sharpe Network Systems Engineer Pacific Wireless Ph +61 3 9584 8966 Mob 0408 009 502 Helpdesk 1300 300 616 email lsharpe@pacificwireless.com.au web www.pacificwireless.com.au -----Original Message----- From: Andy Furniss [mailto:lists@andyfurniss.entadsl.com] Sent: Thursday, March 29, 2007 2:59 AM To: Leigh Sharpe Cc: lartc Subject: Re: IFB setup was no subject Leigh Sharpe wrote:> Hi all, > I''m having a hell of a time getting my IFB to work. I know I''ve done > this before, so I''m missing something stupid. Can anybody tell me what > it might be? > Configs as follows: > > -------- > #!/bin/sh > modprobe ifb numifbs=1000 > modprobe act_mirred > modprobe 8021q > > brctl addbr br0 > brctl setfd br0 0 > brctl stp br0 off > brctl addif br0 eth1 > brctl addif br0 eth2 > ifconfig eth1 up > ifconfig eth2 up > ifconfig br0 up > > tc qdisc add dev eth1 root handle 1:0 cbq bandwidth 100Mbit avpkt 1000 > cell 8 > tc qdisc add dev eth2 root handle 1:0 cbq bandwidth 100Mbit avpkt 1000 > cell 8 > > ifconfig ifb0 up > > tc qdisc add dev ifb0 root handle 1:0 cbq bandwidth 100Mbit avpkt 1000 > cell 8 > > tc class add dev ifb0 parent 1:0 classid 1:1 cbq bandwidth 100Mbit rate > 100Kbit weight 10Kbit prio 5 allot 1514 cell 8 maxburst 20 avpkt 1000 > bounded isolated > > tc qdisc add dev eth1 ingress > tc qdisc add dev eth2 ingress > tc filter add dev eth1 parent ffff: protocol ip prio 10 u32 match u32 0 > 0 action mirred egress redirect dev ifb0 > tc filter add dev eth2 parent ffff: protocol ip prio 10 u32 match u32 0 > 0 action mirred egress redirect dev ifb0 > > tc filter add dev ifb0 parent 1:0 protocol ip prio 1 u32 match u32 0 0 > flowid 1:1 > -------------- > > By my reckoning, this should redirect all packets passing through eth2 > and eth1 through the IFB, and put a rate limit on the IFB of > 100Kbits/sec. > What have I missed?Subject changed as the no subject thread I see is quite long and diverse :-) It will only get ingress ethertype ip traffic on eth 1/2 (I don''t know whether bridging complicates it - still havent fixed my own bridge yet) You need two more ... action mirred egress redirect ... to get egress eth1/2 traffic aswell. You need something classfull on the roots for it to work - you have CBQ already if you didn''t prio would have done. If you have a qdisc on the real interface and redirect egress traffic to ifb then you can''t classify again, though classification done before the redirect will be reinstated after the packets come back from ifb. Generally - I never used CBQ so don''t know it''s limitations/behavior. Accepting this may just be a test, but if you did it like this for real ie. sending in/out traffic to one slow buffer it will make for a truly horrible user experience - you should at least seperate the directions into seperate classes and make the classes share the bandwidth. I wonder if using policers and shared meters would be a nicer alternative - not that I have any examples to hand. If the nics are gig/ you use xen turn off * segmentation offload with ethtool -k Andy.
Leigh Sharpe wrote:> Seems that the example I gave actually works, but not the way I''m using it. > I am bridging VLAN tagged packets,Try protocol 8021q or whatever its number is - if there are other protocol filters you will need a different prio or you will get an error. but for some reason they are not being subjected to the rate limit. If I pass normal, untagged packets through this setup, it behaves as expected. However, once I put tagged packets through the bridge, it fails to shape traffic.> I don''t want to have to use VLAN sub-interfaces, because the VLAN code strips the 802.1q tag from packets before they can be examined, which causes me problems in other areas.Are these wireless customers? I''ve never shaped wireless - do you get alot of extra loss from link layer, what''s the bandwidth, single duplex or is it round robin type? I wonder if htb tweaked/untweaked/hfsc/policers could be better than cbq - you may be able to get things better for link latyer, tcpdumps will show you how bursty things are for users. Andy.
>Try protocol 8021q or whatever its number is - if there are other >protocol filters you will need a different prio or you will get an error.I''ll try that. I hav just seem some older stuff on the VLAN mailing list which might do what I want. I''m still investigating.>Are these wireless customers?Nup. Well, not directly. This is going on our backbone, so I''m not taking traffic straight off the wireless. Ultimately, it will be delivered to a customer over a wireless link, but there''s lots of ethernet between the QOS box and the wireless. By the way, wireless != 802.11, there''s plenty of other flavours which all taste just like ethernet.> I wonder if htb tweaked/untweaked/hfsc/policers could be better than cbqAt this point I''m not locked into any particular discipline, but cbq looked like it would do what I wanted. I''ll look a little further into the other possibilities. Regards, Leigh Leigh Sharpe Network Systems Engineer Pacific Wireless Ph +61 3 9584 8966 Mob 0408 009 502 Helpdesk 1300 300 616 email lsharpe@pacificwireless.com.au web www.pacificwireless.com.au -----Original Message----- From: Andy Furniss [mailto:lists@andyfurniss.entadsl.com] Sent: Thursday, March 29, 2007 12:33 PM To: Leigh Sharpe Cc: lartc Subject: Re: [LARTC] RE: IFB setup was no subject Leigh Sharpe wrote:> Seems that the example I gave actually works, but not the way I''m using it. > I am bridging VLAN tagged packets,Try protocol 8021q or whatever its number is - if there are other protocol filters you will need a different prio or you will get an error. but for some reason they are not being subjected to the rate limit. If I pass normal, untagged packets through this setup, it behaves as expected. However, once I put tagged packets through the bridge, it fails to shape traffic.> I don''t want to have to use VLAN sub-interfaces, because the VLAN code strips the 802.1q tag from packets before they can be examined, which causes me problems in other areas.Are these wireless customers? I''ve never shaped wireless - do you get alot of extra loss from link layer, what''s the bandwidth, single duplex or is it round robin type? I wonder if htb tweaked/untweaked/hfsc/policers could be better than cbq - you may be able to get things better for link latyer, tcpdumps will show you how bursty things are for users. Andy.
Leigh Sharpe wrote:> Nup. Well, not directly. This is going on our backbone, so I''m not taking traffic straight off the wireless. Ultimately, it will be delivered to a customer over a wireless link, but there''s lots of ethernet between the QOS box and the wireless. > By the way, wireless != 802.11, there''s plenty of other flavours which all taste just like ethernet.It was the numifbs 1000 and your sig that made me think wireless customers. I just saw a patch go in for ifb that fixes a potential crash. http://www.spinics.net/lists/netdev/msg28168.html Andy.
>Try protocol 8021q or whatever its number is -Thanks Andy, this did the trick. And now for the next question. 802.1q is protocol number 0x8100. Therefore my filter lines look like this: Tc filter add dev eth3 parent ffff: protocol 0x8100 prio 10 u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb0 What is the u32 matching on? Is it matching on IP headers, or is it matching on the protocol specified, ie the VLAN header? For my particular application, I need to decide which IFB to redirect to based on combinations of both VLAN ID and IP src/dst addresses. Can I specify matches for the VLAN ID here? If so, I would presume that I can then use an Iptables mark to filter on, with that mark based on IP address? (ebtables can''t match the IP address of a tagged packet, unfortunately.) Otherwise, I''m going to have to mark the packets with a VLAN ID using ebtables and then another mark from Iptables based on src/dst IP address. What a sodding nightmare. Regards, Leigh Leigh Sharpe Network Systems Engineer Pacific Wireless Ph +61 3 9584 8966 Mob 0408 009 502 Helpdesk 1300 300 616 email lsharpe@pacificwireless.com.au web www.pacificwireless.com.au -----Original Message----- From: lists@andyfurniss.entadsl.com [mailto:lists@andyfurniss.entadsl.com] Sent: Thursday, March 29, 2007 12:33 PM To: Leigh Sharpe Cc: lartc Subject: Re: [LARTC] RE: IFB setup was no subject Leigh Sharpe wrote:> Seems that the example I gave actually works, but not the way I''m using it. > I am bridging VLAN tagged packets,Try protocol 8021q or whatever its number is - if there are other protocol filters you will need a different prio or you will get an error. but for some reason they are not being subjected to the rate limit. If I pass normal, untagged packets through this setup, it behaves as expected. However, once I put tagged packets through the bridge, it fails to shape traffic.> I don''t want to have to use VLAN sub-interfaces, because the VLAN code strips the 802.1q tag from packets before they can be examined, which causes me problems in other areas.Are these wireless customers? I''ve never shaped wireless - do you get alot of extra loss from link layer, what''s the bandwidth, single duplex or is it round robin type? I wonder if htb tweaked/untweaked/hfsc/policers could be better than cbq - you may be able to get things better for link latyer, tcpdumps will show you how bursty things are for users. Andy.
Leigh Sharpe wrote:> >> Try protocol 8021q or whatever its number is - > > Thanks Andy, this did the trick. And now for the next question. > > 802.1q is protocol number 0x8100. Therefore my filter lines look like this: > > Tc filter add dev eth3 parent ffff: protocol 0x8100 prio 10 u32 match u32 0 0 flowid 1:1 action mirred egress redirect dev ifb0 > > What is the u32 matching on? Is it matching on IP headers, or is it matching on the protocol specified, ie the VLAN header? >It''s the ethertype protocol number in ethernet frames - you can also match it with a u32 offset of -2.> For my particular application, I need to decide which IFB to redirect to based on combinations of both VLAN ID and IP src/dst addresses. Can I specify matches for the VLAN ID here? If so, I would presume that I can then use an Iptables mark to filter on, with that mark based on IP address? (ebtables can''t match the IP address of a tagged packet, unfortunately.) Otherwise, I''m going to have to mark the packets with a VLAN ID using ebtables and then another mark from Iptables based on src/dst IP address. What a sodding nightmare.I''ve never used ebtables - I also haven''t played with xtables yet - infact I''ve barely touched my PC since xmas and am not up on what''s changed yet. If you plan on marking with tc actions you''ll need to be lucky with kernel versions - maybe it''s OK now, I need to get upto date. I think to match vlan tags/ips you will have workout u32 offsets/masks/hex matches etc tcpdump -e and test alot - Ouch. Looks like Patrick McHardy is in the process of changing all the qdiscs to use hi res timers - cool I''ll have to try that aswell. Andy.