Hi, I notice that if two or more existing connections match an ingress policing filter, the input bandwidth does not get evenly divided up between the n connections. Kinda like litters of baby animals, where the stronger babies get more access to the mothers teats and grow up bigger and faster than their siblings. The only workaround that''s working for me is to set explicit ingress policing filters, which match src and dest host and port, so for each filter there can only be exactly one connection which matches. Then, it''s just a matter of evenly dividing up the allocated total input bw amongst these n processes. This works, but it''s not exactly optimal. -- Kind regards David -- leave this line intact so your email gets through my junk mail filter _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
That is normal, policers do not share bandwich, they simply drop packets on match randomly. Probably you are already using index feature. I checked policer source code, and cant understand anything at all, probabaly everyhing is in other modules. From that little what I could understand, seems that index means sharing some parts of data. probably rate counters, this is quite logical, but than I dont undestand its behavior, each policer should see sum of rates, but match on its own rate. In practice it seems to ignore its own rate and match on sum of rates all at once so that one which packet exceed sum of trates will drop its packet. Maybe you could check if it is corerct? Since it is hard to believe that policer is intended to work in such way so. I dont think policers are capable to divide trafic in fair way, they work as simple one fifo. (all policers as single fifo at once) ----- Original Message ----- From: "David McNab" <david@rebirthing.co.nz> To: <lartc@mailman.ds9a.nl> Sent: Sunday, March 14, 2004 1:11 PM Subject: [LARTC] Weird quirk with ingress policing> Hi, > > I notice that if two or more existing connections match an ingress > policing filter, the input bandwidth does not get evenly divided up > between the n connections. > > Kinda like litters of baby animals, where the stronger babies get more > access to the mothers teats and grow up bigger and faster than their > siblings. > > The only workaround that''s working for me is to set explicit ingress > policing filters, which match src and dest host and port, so for each > filter there can only be exactly one connection which matches. Then, > it''s just a matter of evenly dividing up the allocated total input bw > amongst these n processes. This works, but it''s not exactly optimal. > > -- > > Kind regards > David > > -- > > leave this line intact so your email gets through my junk mail filter > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ >_______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
Thanks for that Roy. I kind of gathered that there''s no (sane) way to share bandwidth fairly amongst the connections matching an ingress policing filter, and your investigations have helped to confirm this. So what''s working well for me, in the case of pyshaper, is to determine the number of connections which match a pyshaper class, divide this number into the total downstream bw allocated for that class, and spit out tc commands on a per-connection basis, matching src/dest ip/port, so that for every ingress policing filter, there is exactly one existing connection which matches. So if I''ve allocated 12kbit for total ingress for a pyshaper class, and there are 4 matching connections, pyshaper executes tc commands to set up 4 ingress policers, each with 3kbit rate. If one of the connections drops out, pyshaper clears these policers, and generates 3 ingress policers each with 4kbit rate. Cheers David Roy wrote:> That is normal, policers do not share bandwich, they simply drop packets on > match randomly. > Probably you are already using index feature. > > I checked policer source code, and cant understand anything at all, > probabaly everyhing is in other modules. >>From that little what I could understand, seems that index means sharing > some parts of data. > probably rate counters, this is quite logical, but than I dont undestand its > behavior, > each policer should see sum of rates, but match on its own rate. > > In practice it seems to ignore its own rate and match on sum of rates all at > once > so that one which packet exceed sum of trates will drop its packet. > Maybe you could check if it is corerct? > Since it is hard to believe that policer is intended to work in such way so. > > I dont think policers are capable to divide trafic in fair way, they work as > simple one fifo. > (all policers as single fifo at once) > > > ----- Original Message ----- > From: "David McNab" <david@rebirthing.co.nz> > To: <lartc@mailman.ds9a.nl> > Sent: Sunday, March 14, 2004 1:11 PM > Subject: [LARTC] Weird quirk with ingress policing > > > >>Hi, >> >>I notice that if two or more existing connections match an ingress >>policing filter, the input bandwidth does not get evenly divided up >>between the n connections. >> >>Kinda like litters of baby animals, where the stronger babies get more >>access to the mothers teats and grow up bigger and faster than their >>siblings. >> >>The only workaround that''s working for me is to set explicit ingress >>policing filters, which match src and dest host and port, so for each >>filter there can only be exactly one connection which matches. Then, >>it''s just a matter of evenly dividing up the allocated total input bw >>amongst these n processes. This works, but it''s not exactly optimal. >> >>-- >> >>Kind regards >>David >> >>-- >> >>leave this line intact so your email gets through my junk mail filter >>_______________________________________________ >>LARTC mailing list / LARTC@mailman.ds9a.nl >>http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ >> > > > _______________________________________________ > LARTC mailing list / LARTC@mailman.ds9a.nl > http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/ >-- Kind regards David -- leave this line intact so your email gets through my junk mail filter _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/
David McNab wrote:> Hi, > > I notice that if two or more existing connections match an ingress > policing filter, the input bandwidth does not get evenly divided up > between the n connections. > > Kinda like litters of baby animals, where the stronger babies get more > access to the mothers teats and grow up bigger and faster than their > siblings.That''s tcp not being fair.> > The only workaround that''s working for me is to set explicit ingress > policing filters, which match src and dest host and port, so for each > filter there can only be exactly one connection which matches. Then, > it''s just a matter of evenly dividing up the allocated total input bw > amongst these n processes. This works, but it''s not exactly optimal. >Ingress shaping is alot harder than egress. It is possible to use htb/sfq for ingress by either shaping outbound on your LAN interface (if you have only one) or by using IMQ on your WAN interface. I found this nicer than using an ingress policer, but it''s not perfect. In reality if you want to properly shape ingress from behind the bottleneck you need something to keep state on all the TCP connections, have the ability to change the advertised window to control them and to be aware of what unstoppable packets are due in the future from the WAN so that new TCP connections can be started without causing your queues to move into your ISPs buffer. Even if you did that you are still up against bursty delivery of packets and occilations of TCP caused for example, by ack compression. Andy. _______________________________________________ LARTC mailing list / LARTC@mailman.ds9a.nl http://mailman.ds9a.nl/mailman/listinfo/lartc HOWTO: http://lartc.org/