libvirt users - Jul 2013 - Re: [ovs-discuss] Libvirt "tc ingress qdisc" automatically removed by ovs vlan tag setting, how?

On Wed, Jul 17, 2013 at 6:06 AM, Qiu Yu <unicell@gmail.com>
wrote:
> After some digging in openvswitch code. My wild guess is that vlan tag
> reconfiguring triggered iface_configure_qos (vswitchd/bridge.c), which
> in turn called netdev_set_policing to reset ingress policing rate.
> Although there's no ingress_policing_rate set in my case, existing
> ingress qdisc still remove by default.
The OVS database in general specifies state, not actions.  That is, if
you set no ingress_policing_rate, then OVS takes that to mean that it
should turn off ingress policing, so it does.

I'm surprised that you actually find ingress policing useful.  Our
experience is that it doesn't work well, regardless of how you
configure it.

On Thu, Jul 18, 2013 at 12:15 AM, Ben Pfaff <blp@nicira.com>
wrote:
> On Wed, Jul 17, 2013 at 6:06 AM, Qiu Yu <unicell@gmail.com> wrote:
>> After some digging in openvswitch code. My wild guess is that vlan tag
>> reconfiguring triggered iface_configure_qos (vswitchd/bridge.c), which
>> in turn called netdev_set_policing to reset ingress policing rate.
>> Although there's no ingress_policing_rate set in my case, existing
>> ingress qdisc still remove by default.
>
> The OVS database in general specifies state, not actions.  That is, if
> you set no ingress_policing_rate, then OVS takes that to mean that it
> should turn off ingress policing, so it does.
I see. So ingress policing managed outside of OVS, which is libvirt in
my case, is overridden (no ingress_policing_rate by default, means
turned off) by OVS.

My question, however, is there any workaround, to preserve or inherit
the ingress policing managed outside of OVS after vlan tagging the
device?
> I'm surprised that you actually find ingress policing useful.  Our
> experience is that it doesn't work well, regardless of how you
> configure it.
Well, I'm not aware of other software solution to achieve ingress
throttling purpose. :( For me, it is better than nothing. Any
alternative solution to share? :)

--
Qiu Yu

On Thu, Jul 18, 2013 at 12:30:19AM +0800, Qiu Yu wrote:
> On Thu, Jul 18, 2013 at 12:15 AM, Ben Pfaff <blp@nicira.com> wrote:
> > On Wed, Jul 17, 2013 at 6:06 AM, Qiu Yu <unicell@gmail.com>
wrote:
> >> After some digging in openvswitch code. My wild guess is that vlan
tag
> >> reconfiguring triggered iface_configure_qos (vswitchd/bridge.c),
which
> >> in turn called netdev_set_policing to reset ingress policing rate.
> >> Although there's no ingress_policing_rate set in my case,
existing
> >> ingress qdisc still remove by default.
> >
> > The OVS database in general specifies state, not actions.  That is, if
> > you set no ingress_policing_rate, then OVS takes that to mean that it
> > should turn off ingress policing, so it does.
> 
> I see. So ingress policing managed outside of OVS, which is libvirt in
> my case, is overridden (no ingress_policing_rate by default, means
> turned off) by OVS.
> 
> My question, however, is there any workaround, to preserve or inherit
> the ingress policing managed outside of OVS after vlan tagging the
> device?
OVS doesn't have one now.  We'd take a patch to introduce one.

"VLAN tagging the device" is a bit of a misunderstanding.  This
behavior
will take place on any change to Open vSwitch configuration.  VLAN
tagging is just the one example that you have encountered.

On 17.07.2013 18:30, Qiu Yu wrote:
> On Thu, Jul 18, 2013 at 12:15 AM, Ben Pfaff <blp@nicira.com> wrote:
>> On Wed, Jul 17, 2013 at 6:06 AM, Qiu Yu <unicell@gmail.com>
wrote:
>>> After some digging in openvswitch code. My wild guess is that vlan
tag
>>> reconfiguring triggered iface_configure_qos (vswitchd/bridge.c),
which
>>> in turn called netdev_set_policing to reset ingress policing rate.
>>> Although there's no ingress_policing_rate set in my case,
existing
>>> ingress qdisc still remove by default.
>>
>> The OVS database in general specifies state, not actions.  That is, if
>> you set no ingress_policing_rate, then OVS takes that to mean that it
>> should turn off ingress policing, so it does.
> 
> I see. So ingress policing managed outside of OVS, which is libvirt in
> my case, is overridden (no ingress_policing_rate by default, means
> turned off) by OVS.
> 
> My question, however, is there any workaround, to preserve or inherit
> the ingress policing managed outside of OVS after vlan tagging the
> device?
> 
>> I'm surprised that you actually find ingress policing useful.  Our
>> experience is that it doesn't work well, regardless of how you
>> configure it.
> 
> Well, I'm not aware of other software solution to achieve ingress
> throttling purpose. :( For me, it is better than nothing. Any
> alternative solution to share? :)
There's one. It's called IFB (Intermediate Functional Block). This is
basically a pair of interfaces that act like bidirectional pipe. What is
sent at one end, appears on the other one and vice versa. Advantage is,
you can do actual shaping, not just policing. As IFB name says, you can
insert this element between vNIC from domain and the bridge, and
effectively turn incoming traffic to outgoing. And since linux kernel
doesn't allow us to shape incoming, just outgoing traffic, you can
easily use shaping with IFB. There are, however, couple of reasons why
not to go down this path. Firstly, much higher overhead due to packets
going longer paths through linux networking internals. Secondly, IFB
devices cannot be created on the fly. At the time of IFB module load,
one can specify how many IFB devices shall be created, but this cannot
be changed thereafter.

These are the reasons I've decided to go with 'just' policing.

Michal

The difference between policing and actual shaping is: while in shaping
packets are placed into a buffer and delivered delayed, in policing a
packet which is above the limit is just dropped. IOW, policing just cuts
off the peaks. TCP can deal with both, though.