thr3ads.net - Pkg xen devel - [Pkg-xen-devel] Bug#441249: Bug#441249: xen-hypervisor-3.0.3-1-i386-pae: "Problems using XEN when Quagga is running" [Sep 2007]

If this information is useful, please help other people find it:
Share via:

Mikko Korkalo

2007-Sep-11 12:01 UTC

[Pkg-xen-devel] Bug#441249: Bug#441249: xen-hypervisor-3.0.3-1-i386-pae: "Problems using XEN when Quagga is running"

Hi,
> Can you manually do on the xen interfaces what the scripts would? How about
> doing it on some other interface configured in a similar way?
Toying with the vif-route script, I might have found a workaround for this
issue.
If I disable the ifconfig and ip route commands from vif-route script, and bring
up vif interface by hand later on, everything seems to work.

In other works, bringing up the interface in vif-route script causes it to stay
up when I shutdown the domain.

I'm not sure though if the workaround is stable, as I again experienced a
crash
like before ("kernel BUG at drivers/xen/core/evtchn.c:481!") when
doing a lot
of create/shutdown commands, manually bringing up interfaces, etc. I haven't
found a reliable way to reproduce the crash yet.

I need to do more testing before I can say for sure.

What do you think of this?
> Can you try to insulate it within quagga? Does this happen if you just run
> zebra but not ospfd?
If only zebra is running - still a jammed VIF interface.
If only ospfd is running, it actually fixes the problem. (too bad ospfd is
useless without the zebra daemon)

By the way, if you want I could arrange you an SSH access to the xen1 host.


Best regards,
Mikko Korkalo

Mikko Korkalo

2007-Sep-11 15:48 UTC

head link

[Pkg-xen-devel] Bug#441249: Bug#441249: xen-hypervisor-3.0.3-1-i386-pae: "Problems using XEN when Quagga is running"

Hi again,

First, to prove that the problem still exists with standard vif-route (I
didn't do reboots or anything fancy between these tests)

<--------------
xen1:/etc/xen/scripts# cp vif-route.orig vif-route
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    184.9
xen1:/etc/xen/scripts# xm create test7.cfg
Using config file "/etc/xen/test7.cfg".
Started domain test7
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    186.2
test7                                     11      128     1 -b----      2.8
xen1:/etc/xen/scripts# xm shutdown test7
xen1:/etc/xen/scripts# xm list
Error: Device 0 not connected
Usage: xm list [options] [Domain, ...]

List information about all/some domains.
  -l, --long                     Output all VM details in SXP
  --label                        Include security labels

xen1:/etc/xen/scripts# ip link|grep vif
16: vif11.0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue
xen1:/etc/xen/scripts# /etc/init.d/quagga restart
Stopping Quagga daemons (prio:0): (waiting) .. ospfd zebra (bgpd) (ripd)
(ripngd) (ospf6d) (isisd).
Removing all routes made by zebra.
Nothing to flush.
Loading capability module if not yet done.
Starting Quagga daemons (prio:10): zebra ospfd.
xen1:/etc/xen/scripts# ip link|grep vif
-------------->

Then, I disabled everything extra from vif-route. This shows that the
script is just bringing up the interface. Still the same problem.
<--------------
xen1:/etc/xen/scripts# pico vif-route
xen1:/etc/xen/scripts# cat vif-route
#!/bin/bash

# Hacks:
# - handle_iptable disabled
# - setting proxy_arp disabled
# - ip route add/del commands disabled

dir=$(dirname "$0")
. "$dir/vif-common.sh"

main_ip=$(dom0_ip)

case "$command" in
    online)
        ifconfig ${vif} ${main_ip} netmask 255.255.255.255 up
        #echo 1 >/proc/sys/net/ipv4/conf/${vif}/proxy_arp
        ipcmd='add'
        cmdprefix=''
        ;;
    offline)
        do_without_error ifdown ${vif}
        ipcmd='del'
        cmdprefix='do_without_error'
        ;;
esac

#if [ "${ip}" ] ; then
#    # If we've been given a list of IP addresses, then add routes from
dom0 to
#    # the guest using those addresses.
#    for addr in ${ip} ; do
#      ${cmdprefix} ip route ${ipcmd} ${addr} dev ${vif} src ${main_ip}
#    done
#fi
#
#handle_iptable

log debug "Successful vif-route $command for $vif."
if [ "$command" == "online" ]
then
  success
fi
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    188.0
xen1:/etc/xen/scripts# xm create test7.cfg
Using config file "/etc/xen/test7.cfg".
Started domain test7
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    189.3
test7                                     12      128     1 -b----      2.8
xen1:/etc/xen/scripts# xm shutdown test7
xen1:/etc/xen/scripts# ip link|grep vif
17: vif12.0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc noqueue
xen1:/etc/xen/scripts# ip link|grep vif
17: vif12.0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue
xen1:/etc/xen/scripts# xm list
Error: Device 0 not connected
Usage: xm list [options] [Domain, ...]

List information about all/some domains.
  -l, --long                     Output all VM details in SXP
  --label                        Include security labels

xen1:/etc/xen/scripts# /etc/init.d/quagga restart
Stopping Quagga daemons (prio:0): (waiting) .. ospfd (waiting) . zebra
(bgpd) (ripd) (ripngd) (ospf6d) (isisd).
Removing all routes made by zebra.
Nothing to flush.
Loading capability module if not yet done.
Starting Quagga daemons (prio:10): zebra ospfd.
xen1:/etc/xen/scripts# ip link|grep vif
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    190.4
xen1:/etc/xen/scripts#
-------------->


Then, I also disable ifconfig command from vif-route. Problem is gone.
<--------------
xen1:/etc/xen/scripts# pico vif-route
xen1:/etc/xen/scripts# cat vif-route
#!/bin/bash

# Hacks:
# - handle_iptable disabled
# - setting proxy_arp disabled
# - ip route add/del commands disabled
# - ifconfig up command disabled

dir=$(dirname "$0")
. "$dir/vif-common.sh"

main_ip=$(dom0_ip)

case "$command" in
    online)
        #ifconfig ${vif} ${main_ip} netmask 255.255.255.255 up
        #echo 1 >/proc/sys/net/ipv4/conf/${vif}/proxy_arp
        ipcmd='add'
        cmdprefix=''
        ;;
    offline)
        do_without_error ifdown ${vif}
        ipcmd='del'
        cmdprefix='do_without_error'
        ;;
esac

#if [ "${ip}" ] ; then
#    # If we've been given a list of IP addresses, then add routes from
dom0 to
#    # the guest using those addresses.
#    for addr in ${ip} ; do
#      ${cmdprefix} ip route ${ipcmd} ${addr} dev ${vif} src ${main_ip}
#    done
#fi
#
#handle_iptable

log debug "Successful vif-route $command for $vif."
if [ "$command" == "online" ]
then
  success
fi
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    190.8
xen1:/etc/xen/scripts# xm create test7.cfg
Using config file "/etc/xen/test7.cfg".
Started domain test7
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    192.0
test7                                     13      128     1 -b----      2.8
xen1:/etc/xen/scripts# xm shutdown test7
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    192.8
xen1:/etc/xen/scripts# ip link|grep vif
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    192.9
xen1:/etc/xen/scripts#
-------------->


Now the same as before, except I bring up the interface by hand. No problem.
<--------------
xen1:/etc/xen/scripts# cat vif-route
#!/bin/bash

# Hacks:
# - handle_iptable disabled
# - setting proxy_arp disabled
# - ip route add/del commands disabled
# - ifconfig up command disabled

dir=$(dirname "$0")
. "$dir/vif-common.sh"

main_ip=$(dom0_ip)

case "$command" in
    online)
        #ifconfig ${vif} ${main_ip} netmask 255.255.255.255 up
        #echo 1 >/proc/sys/net/ipv4/conf/${vif}/proxy_arp
        ipcmd='add'
        cmdprefix=''
        ;;
    offline)
        do_without_error ifdown ${vif}
        ipcmd='del'
        cmdprefix='do_without_error'
        ;;
esac

#if [ "${ip}" ] ; then
#    # If we've been given a list of IP addresses, then add routes from
dom0 to
#    # the guest using those addresses.
#    for addr in ${ip} ; do
#      ${cmdprefix} ip route ${ipcmd} ${addr} dev ${vif} src ${main_ip}
#    done
#fi
#
#handle_iptable

log debug "Successful vif-route $command for $vif."
if [ "$command" == "online" ]
then
  success
fi
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    196.2
xen1:/etc/xen/scripts# xm create test7.cfg
Using config file "/etc/xen/test7.cfg".
Started domain test7
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    197.5
test7                                     15      128     1 -b----      2.8
xen1:/etc/xen/scripts# ip link|grep vif
20: vif15.0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop
xen1:/etc/xen/scripts# ifconfig vif15.0 172.16.255.11 netmask
255.255.255.255 up
xen1:/etc/xen/scripts# ip link|grep vif
20: vif15.0: <BROADCAST,MULTICAST,UP,10000> mtu 1500 qdisc noqueue
xen1:/etc/xen/scripts# xm shutdown test7
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    197.7
test7                                     15      128     1 -b----      4.4
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    198.4
xen1:/etc/xen/scripts# ip link|grep vif
xen1:/etc/xen/scripts# xm create test7.cfg
Using config file "/etc/xen/test7.cfg".
Started domain test7
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    199.7
test7                                     16      128     1 -b----      2.8
xen1:/etc/xen/scripts#
-------------->

The same as before, but I also add routes and set proxy_arp on, meaning
a working network for the guest. No problems either after shutdown.
<--------------
xen1:/etc/xen/scripts# cat vif-route
#!/bin/bash

# Hacks:
# - handle_iptable disabled
# - setting proxy_arp disabled
# - ip route add/del commands disabled
# - ifconfig up command disabled

dir=$(dirname "$0")
. "$dir/vif-common.sh"

main_ip=$(dom0_ip)

case "$command" in
    online)
        #ifconfig ${vif} ${main_ip} netmask 255.255.255.255 up
        #echo 1 >/proc/sys/net/ipv4/conf/${vif}/proxy_arp
        ipcmd='add'
        cmdprefix=''
        ;;
    offline)
        do_without_error ifdown ${vif}
        ipcmd='del'
        cmdprefix='do_without_error'
        ;;
esac

#if [ "${ip}" ] ; then
#    # If we've been given a list of IP addresses, then add routes from
dom0 to
#    # the guest using those addresses.
#    for addr in ${ip} ; do
#      ${cmdprefix} ip route ${ipcmd} ${addr} dev ${vif} src ${main_ip}
#    done
#fi
#
#handle_iptable

log debug "Successful vif-route $command for $vif."
if [ "$command" == "online" ]
then
  success
fi
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    203.2
xen1:/etc/xen/scripts# xm create test7.cfg
Using config file "/etc/xen/test7.cfg".
Started domain test7
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    204.5
test7                                     18      128     1 -b----      2.8
xen1:/etc/xen/scripts# ip link|grep vif
23: vif18.0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop
xen1:/etc/xen/scripts# ifconfig vif18.0 172.16.255.11 netmask
255.255.255.255 up
xen1:/etc/xen/scripts# echo 1 >/proc/sys/net/ipv4/conf/vif18.0/proxy_arp
xen1:/etc/xen/scripts# ip route add 172.16.255.44 dev vif18.0
xen1:/etc/xen/scripts# ping 172.16.255.44 -c 1
PING 172.16.255.44 (172.16.255.44) 56(84) bytes of data.
64 bytes from 172.16.255.44: icmp_seq=1 ttl=64 time=782 ms

--- 172.16.255.44 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 782.787/782.787/782.787/0.000 ms
xen1:/etc/xen/scripts# ping 172.16.255.44 -c 1
PING 172.16.255.44 (172.16.255.44) 56(84) bytes of data.
64 bytes from 172.16.255.44: icmp_seq=1 ttl=64 time=0.037 ms

--- 172.16.255.44 ping statistics ---
1 packets transmitted, 1 received, 0% packet loss, time 0ms
rtt min/avg/max/mdev = 0.037/0.037/0.037/0.000 ms
xen1:/etc/xen/scripts# xm shutdown test7
xen1:/etc/xen/scripts# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1891     2 r-----    205.4
xen1:/etc/xen/scripts# ip link|grep vif
xen1:/etc/xen/scripts#
-------------->

Oddly enough, if I run these commands from within the vif-route script
in background, with a 10 second delay before they are executed, I get
the same problem (as if something in the environment was different when
running them by hand).

Best Regards,
Mikko Korkalo

Guido Trotter kirjoitti:> On Tue, Sep 11, 2007 at 03:01:57PM +0300, Mikko Korkalo wrote:
>
> Hi,
>
>   
>>> Can you manually do on the xen interfaces what the scripts would?
How about
>>> doing it on some other interface configured in a similar way?
>>>       
>> Toying with the vif-route script, I might have found a workaround for
this
>> issue.
>> If I disable the ifconfig and ip route commands from vif-route script,
and bring
>> up vif interface by hand later on, everything seems to work.
>>
>> In other works, bringing up the interface in vif-route script causes it
to stay
>> up when I shutdown the domain.
>>
>>     
>
> This is strange, as vif-route should just be doing what you're doing
manually
> later... Are you sure you're then doing exactly the same steps?
>
>   
>> I'm not sure though if the workaround is stable, as I again
experienced a crash
>> like before ("kernel BUG at drivers/xen/core/evtchn.c:481!")
when doing a lot
>> of create/shutdown commands, manually bringing up interfaces, etc. I
haven't
>> found a reliable way to reproduce the crash yet.
>>
>>     
>
> This is an unrelated problem... It depends from the xen patch you're
using and
> should disappear if you update it!
>
>   
>> If only zebra is running - still a jammed VIF interface.
>> If only ospfd is running, it actually fixes the problem. (too bad ospfd
is
>> useless without the zebra daemon)
>>
>>     
>
> Thanks!
>
> Guido
>
>
>

Mikko Korkalo

2007-Sep-12 07:29 UTC

head link

[Pkg-xen-devel] Bug#441249: Bug#441249: xen-hypervisor-3.0.3-1-i386-pae: "Problems using XEN when Quagga is running"

Hello,

I disabled ipv6: no more problems, whatever vif-script I use. I don't need
ipv6
so this workaround works fine for me.

This issue is maybe unrelated to Xen. I would need to do more testing to be
sure.

I hope this will save some time from someone else getting the same problem. :)

Here's what I did:
<---------
xen1:~# pico /etc/modprobe.d/00disable-ipv6
xen1:~# cat /etc/modprobe.d/00disable-ipv6
alias net-pf-10 off
alias ipv6 off
xen1:~# reboot
...
xen1:~# cp /etc/xen/scripts/vif-route.orig /etc/xen/scripts/vif-route
xen1:~# xm create test7.cfg
Using config file "/etc/xen/test7.cfg".
Started domain test7
xen1:~# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1892     2 r-----      9.8
test7                                      1      128     1 -b----      2.8
xen1:~# xm shutdown test7
xen1:~# xm list
Name                                      ID Mem(MiB) VCPUs State   Time(s)
Domain-0                                   0     1892     2 r-----     10.6
xen1:~# xm create test7.cfg
Using config file "/etc/xen/test7.cfg".
Started domain test7
xen1:~#
--------->


Best regards,
Mikko Korkalo

Apparently Analagous Threads

Search for more possibly parallel threads

Pkg xen devel - Sep 2007 - Bug#441249: Bug#441249: xen-hypervisor-3.0.3-1-i386-pae: "Problems using XEN when Quagga is running"

[Pkg-xen-devel] Bug#441249: Bug#441249: xen-hypervisor-3.0.3-1-i386-pae: "Problems using XEN when Quagga is running"

[Pkg-xen-devel] Bug#441249: Bug#441249: xen-hypervisor-3.0.3-1-i386-pae: "Problems using XEN when Quagga is running"

[Pkg-xen-devel] Bug#441249: Bug#441249: xen-hypervisor-3.0.3-1-i386-pae: "Problems using XEN when Quagga is running"

Apparently Analagous Threads