libvirt users - Jul 2013 - Libvirt "tc ingress qdisc" automatically removed by ovs vlan tag setting, how?

With outbound QoS setting in Libvirt XML, libvirt will add a tc
ingress qdisc for traffic shaping. Then if you set VLAN tag to that
tap device, this qdisc will automatically gone by no reason.

Could anyone shed some lights where should I look into? I'm really
confused and got no clue here. Thanks!

Steps to reproduce
--
# virsh start instance-name
# virsh dumpxml instance-name

...
    <interface type='bridge'>
      <mac address='fa:16:3e:b9:8f:2a'/>
      <source bridge='br-int'/>
      <virtualport type='openvswitch'>
        <parameters
interfaceid='0a2b02ca-4824-4bda-baa9-05fff7a3146d'/>
      </virtualport>
      <target dev='tap0a2b02ca-48'/>
      <model type='virtio'/>
      <bandwidth>
        <outbound average='256' peak='512'
burst='512'/>
      </bandwidth>
      <alias name='net0'/>
      <address type='pci' domain='0x0000' bus='0x00'
slot='0x03'
function='0x0'/>
    </interface>
...

# tc qdisc ls dev tap0a2b02ca-48
qdisc htb 1: root refcnt 2 r2q 10 default 1 direct_packets_stat 0
qdisc sfq 2: parent 1:1 limit 127p quantum 1514b perturb 10sec
qdisc ingress ffff: parent ffff:fff1 ----------------  (<-- already showed
up)

# ovs-vsctl set port tap0a2b02ca-48 tag=1
# tc qdisc ls dev tap0a2b02ca-48
qdisc htb 1: root refcnt 2 r2q 10 default 1 direct_packets_stat 0
qdisc sfq 2: parent 1:1 limit 127p quantum 1514b perturb 10sec
(<-- ingress qdisc is gone)

After that, if add this qdisc manually and re-set vlan tag, problem no
longer exists. But it can always be reproduced after "virsh destroy /
start" cycle.

# tc qdisc add dev tap0a2b02ca-48 handle ffff: ingress
# ovs-vsctl clear port tap0a2b02ca-48 tag
# ovs-vsctl set port tap0a2b02ca-48 tag=1
# tc qdisc ls dev tap0a2b02ca-48
qdisc htb 1: root refcnt 2 r2q 10 default 1 direct_packets_stat 0
qdisc sfq 2: parent 1:1 limit 127p quantum 1514b perturb 10sec
qdisc ingress ffff: parent ffff:fff1 ----------------  (<-- still exists)

Other information
--
# ovs-vsctl --version
ovs-vsctl (Open vSwitch) 1.9.2
Compiled May 27 2013 14:19:16
# uname -r
2.6.32-358.111.1.openstack.el6.x86_64
# virsh --version (This is a version I built from libvirt upstream
origin/v0.9.11-stable branch)
0.9.11.9

--
Qiu Yu

After some digging in openvswitch code. My wild guess is that vlan tag
reconfiguring triggered iface_configure_qos (vswitchd/bridge.c), which
in turn called netdev_set_policing to reset ingress policing rate.
Although there's no ingress_policing_rate set in my case, existing
ingress qdisc still remove by default.

Could some openvswitch guy help to confirm and suggest how to fix or
workaround? It seems to be a bug of incompatible with existing ingress
qdisc settings.

Thanks,
--
Qiu Yu


On Wed, Jul 17, 2013 at 8:09 PM, Qiu Yu <unicell@gmail.com>
wrote:
> With outbound QoS setting in Libvirt XML, libvirt will add a tc
> ingress qdisc for traffic shaping. Then if you set VLAN tag to that
> tap device, this qdisc will automatically gone by no reason.
>
> Could anyone shed some lights where should I look into? I'm really
> confused and got no clue here. Thanks!
>
> Steps to reproduce
> --
> # virsh start instance-name
> # virsh dumpxml instance-name
>
> ...
>     <interface type='bridge'>
>       <mac address='fa:16:3e:b9:8f:2a'/>
>       <source bridge='br-int'/>
>       <virtualport type='openvswitch'>
>         <parameters
interfaceid='0a2b02ca-4824-4bda-baa9-05fff7a3146d'/>
>       </virtualport>
>       <target dev='tap0a2b02ca-48'/>
>       <model type='virtio'/>
>       <bandwidth>
>         <outbound average='256' peak='512'
burst='512'/>
>       </bandwidth>
>       <alias name='net0'/>
>       <address type='pci' domain='0x0000'
bus='0x00' slot='0x03'
> function='0x0'/>
>     </interface>
> ...
>
> # tc qdisc ls dev tap0a2b02ca-48
> qdisc htb 1: root refcnt 2 r2q 10 default 1 direct_packets_stat 0
> qdisc sfq 2: parent 1:1 limit 127p quantum 1514b perturb 10sec
> qdisc ingress ffff: parent ffff:fff1 ----------------  (<-- already
showed up)
>
> # ovs-vsctl set port tap0a2b02ca-48 tag=1
> # tc qdisc ls dev tap0a2b02ca-48
> qdisc htb 1: root refcnt 2 r2q 10 default 1 direct_packets_stat 0
> qdisc sfq 2: parent 1:1 limit 127p quantum 1514b perturb 10sec
> (<-- ingress qdisc is gone)
>
> After that, if add this qdisc manually and re-set vlan tag, problem no
> longer exists. But it can always be reproduced after "virsh destroy /
> start" cycle.
>
> # tc qdisc add dev tap0a2b02ca-48 handle ffff: ingress
> # ovs-vsctl clear port tap0a2b02ca-48 tag
> # ovs-vsctl set port tap0a2b02ca-48 tag=1
> # tc qdisc ls dev tap0a2b02ca-48
> qdisc htb 1: root refcnt 2 r2q 10 default 1 direct_packets_stat 0
> qdisc sfq 2: parent 1:1 limit 127p quantum 1514b perturb 10sec
> qdisc ingress ffff: parent ffff:fff1 ----------------  (<-- still
exists)
>
> Other information
> --
> # ovs-vsctl --version
> ovs-vsctl (Open vSwitch) 1.9.2
> Compiled May 27 2013 14:19:16
> # uname -r
> 2.6.32-358.111.1.openstack.el6.x86_64
> # virsh --version (This is a version I built from libvirt upstream
> origin/v0.9.11-stable branch)
> 0.9.11.9
>
> --
> Qiu Yu

On Wed, Jul 17, 2013 at 6:06 AM, Qiu Yu <unicell@gmail.com>
wrote:
> After some digging in openvswitch code. My wild guess is that vlan tag
> reconfiguring triggered iface_configure_qos (vswitchd/bridge.c), which
> in turn called netdev_set_policing to reset ingress policing rate.
> Although there's no ingress_policing_rate set in my case, existing
> ingress qdisc still remove by default.
The OVS database in general specifies state, not actions.  That is, if
you set no ingress_policing_rate, then OVS takes that to mean that it
should turn off ingress policing, so it does.

I'm surprised that you actually find ingress policing useful.  Our
experience is that it doesn't work well, regardless of how you
configure it.