Xen users - Jan 2006 - Infinite loop shutting down xendomains

Pardon me for a long detailed post but this looks pretty serious.  The
bottom line is that, when I attempt to shut down or reboot my simple
xen-2.0.7 server running FC3 with two domUs, shutting down xendomains
appears to send python into an infinite loop.  This may be due to a
serious misconfiguration on my part but it is certainly a big problem.

If we shutdown the domUs manually first, the system reboots perfectly
fine.

I''ll describe some history and include lots of log information.  The
configuration is slightly unusual.  Both domUs are VPN gateways running
the 2.4.30 kernel with openswan and openvpn.  Because of the openvpn,
but are privileged domains but the only device driver configured in the
kernel is the openvpn tun/tap adapter.

We first noticed problems during shutdown in that the last domU to
shutdown crashed very nastily when closing openvpn.  We could see this
even when manually stopping the domUs.  We surmised that the problem was
we had tun0 and tun1 defined in both domains and killing them in one
must be killing them in another.  We renamed those tun adapters in the
second domain to tun10 and tun11 and that problem went away.

We still noticed very long reboot times.  The hang seemed to be at
killall but the "logjam" seemed to break after xendomains errored out.
We assumed this was because killall killed something that xendomains
needed to shutdown gracefully so we added explicit K scripts in rc0.d
and rc6.d for xend and xendomains.

At first this seemed to work but it must have been only coincidental
because the problem returned.  We then realized we had brain cramped and
shutdown xend before xendomains.  We thought for sure we had found the
problem and put them in the proper order but, alas, the problem did not
go away.

To gain a little more insight into exactly what was happening, we
manually ran each kill script in rc6.d.  When we ran K78xendomains stop,
we saw the problem, or at least many aspect of it.
>From a xen console, we saw that the first domU had cleanly shutdown but
the second one was still listed howbeit with zero memory.

The hard disks were thrashing madly and top showed python consuming 99.9
percent CPU utilization.  Most of the time, during reboot, we do not see
the madly thrashing disks.  They do so for a moment as one might expect
when shutting down the domains but then just blip occasionally
thereafter.

Here is what we see in the logs:
>From messages (ntpd was the last service stopped before xendomains):
Jan 28 22:05:49 xenpub1dc1 ntpd[3652]: ntpd exiting on signal 15
Jan 28 22:05:49 xenpub1dc1 ntpd: ntpd shutdown succeeded
Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.0 left promiscuous mode
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
disabled state
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
disabled state
Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.1 left promiscuous mode
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
disabled state
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
disabled state
Jan 28 22:06:23 xenpub1dc1 net.agent[4714]: remove event not handled
Jan 28 22:06:23 xenpub1dc1 net.agent[4723]: remove event not handled
Jan 28 22:06:23 xenpub1dc1 net.agent[4737]: remove event not handled
Jan 28 22:06:23 xenpub1dc1 net.agent[4746]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.0 left promiscuous mode
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.1 left promiscuous mode
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 net.agent[4774]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 net.agent[4766]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 net.agent[4789]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 net.agent[4798]: remove event not handled

Here is the complete xend-debug.log:
network start bridge=xen-br0 netdev=eth0 antispoof=no
VIRTUAL MEMORY ARRANGEMENT:
 Loaded kernel: c0100000->c0319394
 Init. ramdisk: c031a000->c031a000
 Phys-Mach map: c031a000->c0332000
 Page tables:   c0332000->c0334000
 Start info:    c0334000->c0335000
 Boot stack:    c0335000->c0336000
 TOTAL:         c0000000->c0400000
 ENTRY ADDRESS: c0100000
vif-bridge up vif=vif1.0 domain=niagras1 mac=02:00:00:01:02:00
bridge=xen-br0
vif-bridge up vif=vif1.1 domain=niagras1 mac=aa:00:00:58:4a:a7
bridge=xen-br0
VIRTUAL MEMORY ARRANGEMENT:
 Loaded kernel: c0100000->c0319394
 Init. ramdisk: c031a000->c031a000
 Phys-Mach map: c031a000->c0332000
 Page tables:   c0332000->c0334000
 Start info:    c0334000->c0335000
 Boot stack:    c0335000->c0336000
 TOTAL:         c0000000->c0400000
 ENTRY ADDRESS: c0100000
vif-bridge up vif=vif2.0 domain=vpn1dc1 mac=02:00:00:01:01:00
bridge=xen-br0
vif-bridge up vif=vif2.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
recv_fe_driver_status> {''status'': 1}

recv_fe_driver_status>

recv_fe_interface_connect {''tx_shmem_frame'': 44763,
''rx_shmem_frame'':
44764, ''handle'': 0}

recv_fe_interface_connect {''tx_shmem_frame'': 44767,
''rx_shmem_frame'':
44768, ''handle'': 1}
recv_fe_driver_status> {''status'': 1}

recv_fe_driver_status>

recv_fe_interface_connect {''tx_shmem_frame'': 69339,
''rx_shmem_frame'':
69340, ''handle'': 0}

recv_fe_interface_connect {''tx_shmem_frame'': 69343,
''rx_shmem_frame'':
69344, ''handle'': 1}
{}
{}
VirqClient.virqReceived> 4
vif-bridge down vif=vif2.0 domain=vpn1dc1 mac=02:00:00:01:01:00
bridge=xen-br0
vif-bridge down vif=vif2.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
VirqClient.virqReceived> 4
vif-bridge down vif=vif1.0 domain=niagras1 mac=02:00:00:01:02:00
bridge=xen-br0
vif-bridge down vif=vif1.1 domain=niagras1 mac=aa:00:00:26:24:c4
bridge=xen-br0
Traceback (most recent call last):
  File "/usr/lib/python2.3/site-packages/twisted/protocols/basic.py",
line 229, in dataReceived
    return self.rawDataReceived(data)
  File "/usr/lib/python2.3/site-packages/twisted/protocols/http.py",
line 1015, in rawDataReceived
    self.allContentReceived()
  File "/usr/lib/python2.3/site-packages/twisted/protocols/http.py",
line 1006, in allContentReceived
    req.requestReceived(command, path, version)
  File "/usr/lib/python2.3/site-packages/twisted/protocols/http.py",
line 557, in requestReceived
    self.process()
--- <exception caught here> ---
  File "/usr/lib/python2.3/site-packages/twisted/web/server.py", line
165, in process
    self.render(resrc)
  File "/usr/lib/python2.3/site-packages/twisted/web/server.py", line
172, in render
    body = resrc.render(self)
  File "/usr/lib/python2.3/site-packages/twisted/web/resource.py",
line
201, in render
    return m(request)
  File "/usr/lib/python/xen/xend/server/SrvDomain.py", line 180, in
render_POST
    return self.perform(req)
  File "/usr/lib/python/xen/xend/server/SrvBase.py", line 103, in
perform
    return self._perform(op, op_method, req)
  File "/usr/lib/python/xen/xend/server/SrvBase.py", line 109, in
_perform
    return self._perform_err(err, op, req)
  File "/usr/lib/python/xen/xend/server/SrvBase.py", line 107, in
_perform
    val = op_method(op, req)
  File "/usr/lib/python/xen/xend/server/SrvDomain.py", line 47, in
op_shutdown
    val = fn(req.args, {''dom'': self.dom.id})
  File "/usr/lib/python/xen/xend/Args.py", line 145, in __call__
    return self.call_with_form_args(self.fn, fargs, xargs=xargs)
  File "/usr/lib/python/xen/xend/Args.py", line 121, in
call_with_form_args
    return fn(*params, **keys)
  File "/usr/lib/python/xen/xend/XendDomain.py", line 477, in
domain_shutdown
    val = xend.domain_shutdown(dominfo.id, reason, key)
  File "/usr/lib/python/xen/xend/server/SrvDaemon.py", line 723, in
domain_shutdown
    ctrl = self.domainCF.getController(dom)
  File "/usr/lib/python/xen/xend/server/controller.py", line 323, in
getController
    ctrl = self.createController(dom)
  File "/usr/lib/python/xen/xend/server/domain.py", line 20, in
createController
    return DomainController(self, dom)
  File "/usr/lib/python/xen/xend/server/domain.py", line 38, in
__init__
    self.registerChannel()
  File "/usr/lib/python/xen/xend/server/controller.py", line 220, in
registerChannel
    self.channel = self.channelFactory.domChannel(self.dom)
  File "/usr/lib/python/xen/xend/server/channel.py", line 83, in
domChannel
    remote_port=remote_port)
  File "/usr/lib/python/xen/xend/server/channel.py", line 253, in
__init__
    remote_port=remote_port)
  File "/usr/lib/python/xen/xend/server/channel.py", line 137, in
createPort
    remote_port=int(remote_port))
xen.lowlevel.xu.PortError: Failed to map domain control interface
/usr/lib/python2.3/site-packages/twisted/web/util.py:290: FutureWarning:
hex()/oct() of negative int will return a signed string in Python 2.4
and up
  return ''''''<div
class="instance"><span class="instanceName">%s
instance
@ %s</span>
/usr/lib/python2.3/site-packages/twisted/web/util.py:266: FutureWarning:
hex()/oct() of negative int will return a signed string in Python 2.4
and up
  w(''<div class="dict"><span
class="heading">Dictionary instance @ %
s</span>'' % hex(id(d)))
/usr/lib/python2.3/site-packages/twisted/web/util.py:279: FutureWarning:
hex()/oct() of negative int will return a signed string in Python 2.4
and up
  w(''<div class="list"><span
class="heading">List instance @ %s</span>''
% hex(id(l)))

 Init. ramdisk: c031a000->c031a000
 Phys-Mach map: c031a000->c0332000
 Page tables:   c0332000->c0334000
 Start info:    c0334000->c0335000
 Boot stack:    c0335000->c0336000
 TOTAL:         c0000000->c0400000
 ENTRY ADDRESS: c0100000
vif-bridge up vif=vif7.0 domain=vpn1dc1 mac=02:00:00:01:01:00
bridge=xen-br0
vif-bridge up vif=vif7.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
recv_fe_driver_status> {''status'': 1}

recv_fe_driver_status>

recv_fe_interface_connect {''tx_shmem_frame'': 88257,
''rx_shmem_frame'':
88258, ''handle'': 0}

recv_fe_interface_connect {''tx_shmem_frame'': 88261,
''rx_shmem_frame'':
88262, ''handle'': 1}
VIRTUAL MEMORY ARRANGEMENT:
 Loaded kernel: c0100000->c0319394
 Init. ramdisk: c031a000->c031a000
 Phys-Mach map: c031a000->c0332000
 Page tables:   c0332000->c0334000
 Start info:    c0334000->c0335000
 Boot stack:    c0335000->c0336000
 TOTAL:         c0000000->c0400000
 ENTRY ADDRESS: c0100000
vif-bridge up vif=vif8.0 domain=niagras1 mac=02:00:00:01:02:00
bridge=xen-br0
vif-bridge up vif=vif8.1 domain=niagras1 mac=aa:00:00:53:7a:5b
bridge=xen-br0
recv_fe_driver_status> {''status'': 1}

recv_fe_driver_status>

recv_fe_interface_connect {''tx_shmem_frame'': 185025,
''rx_shmem_frame'':
185026, ''handle'': 0}

recv_fe_interface_connect {''tx_shmem_frame'': 185029,
''rx_shmem_frame'':
185030, ''handle'': 1}
{}
{}
VirqClient.virqReceived> 4
vif-bridge down vif=vif8.0 domain=niagras1 mac=02:00:00:01:02:00
bridge=xen-br0
vif-bridge down vif=vif8.1 domain=niagras1 mac=aa:00:00:53:7a:5b
bridge=xen-br0
VirqClient.virqReceived> 4
vif-bridge down vif=vif7.0 domain=vpn1dc1 mac=02:00:00:01:01:00
bridge=xen-br0
vif-bridge down vif=vif7.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
-bridge down vif=vif4.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
interface vif4.1 does not exist!
VIRTUAL MEMORY ARRANGEMENT:
 Loaded kernel: c0100000->c02c9df4
 Init. ramdisk: c02ca000->c02ca000
 Phys-Mach map: c02ca000->c02e2000
 Page tables:   c02e2000->c02e4000
 Start info:    c02e4000->c02e5000
 Boot stack:    c02e5000->c02e6000
 TOTAL:         c0000000->c0400000
 ENTRY ADDRESS: c0100000
vif-bridge up vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
bridge=xen-br0
interface vif5.0 does not exist!
vif-bridge up vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
interface vif5.1 does not exist!
recv_fe_driver_status> {''status'': 1}

recv_fe_driver_status>

recv_fe_interface_connect {''tx_shmem_frame'': 202459,
''rx_shmem_frame'':
202460, ''handle'': 0}

recv_fe_interface_connect {''tx_shmem_frame'': 202463,
''rx_shmem_frame'':
202464, ''handle'': 1}
VirqClient.virqReceived> 4
vif-bridge down vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
bridge=xen-br0
interface vif5.0 does not exist!
vif-bridge down vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
interface vif5.1 does not exist!

Here is the information from xend.log:
Jan 28 22:05:49 xenpub1dc1 ntpd[3652]: ntpd exiting on signal 15
Jan 28 22:05:49 xenpub1dc1 ntpd: ntpd shutdown succeeded
Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.0 left promiscuous mode
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
disabled state
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
disabled state
Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.1 left promiscuous mode
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
disabled state
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
disabled state
Jan 28 22:06:23 xenpub1dc1 net.agent[4714]: remove event not handled
Jan 28 22:06:23 xenpub1dc1 net.agent[4723]: remove event not handled
Jan 28 22:06:23 xenpub1dc1 net.agent[4737]: remove event not handled
Jan 28 22:06:23 xenpub1dc1 net.agent[4746]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.0 left promiscuous mode
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.1 left promiscuous mode
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 net.agent[4774]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 net.agent[4766]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 net.agent[4789]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 net.agent[4798]: remove event not handled
Jan 28 22:18:03 xenpub1dc1 shutdown: shutting down for system reboot
Jan 28 22:18:03 xenpub1dc1 init: Switching to runlevel: 6
Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3781]: session closed for
user root
Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3779]: session closed for
user root
Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3780]: session closed for
user root
Jan 28 22:18:04 xenpub1dc1 xendomains: Shutting down all Xen domains:
Jan 28 22:18:05 xenpub1dc1 xendomains: Error: Internal Server Error
Jan 28 22:18:05 xenpub1dc1 xendomains:  failed
Jan 28 22:18:05 xenpub1dc1 xendomains:
Jan 28 22:18:05 xenpub1dc1 rc: Stopping xendomains:  failed
Jan 28 22:18:05 xenpub1dc1 kernel: Kernel logging (proc) stopped.
Jan 28 22:18:05 xenpub1dc1 kernel: Kernel log daemon terminating.
Jan 28 22:18:06 xenpub1dc1 syslog: klogd shutdown succeeded
Jan 28 22:18:06 xenpub1dc1 exiting on signal 15
Jan 28 22:20:06 xenpub1dc1 syslogd 1.4.1: restart.
Jan 28 22:20:06 xenpub1dc1 syslog: syslogd startup succeeded
Jan 28 22:20:06 xenpub1dc1 kernel: klogd 1.4.1, log source = /proc/kmsg
started.
Jan 28 22:20:06 xenpub1dc1 kernel: Linux version 2.6.11.12-xen0
(root@xenpub1dc1.atlasgroup.net) (gcc version 3.4.4 20050721 (Red Hat
3.4.4-2)) #4 Thu Jan
File messages not changed so no update needed.
[root@xenpub1dc1 log]# ls
acpid            boot.log.1  cron.2   mail       messages    prelink.log
samba     spooler    wtmp            xfrd.log
anaconda.log     boot.log.2  cups     maillog    messages.1  rpmpkgs
secure    spooler.1  xend-debug.log  Xorg.0.log
anaconda.syslog  cron        dmesg    maillog.1  messages.2  rpmpkgs.1
secure.1  spooler.2  xend.log        Xorg.0.log.old
boot.log         cron.1      lastlog  maillog.2  ppp         rpmpkgs.2
secure.2  vbox       xend.log.1      yum.log
[root@xenpub1dc1 log]# joe xend-debug.log
Processing ''/etc/joe/joerc''...done
Processing ''/etc/joe/joerc''...done









































    I    xend-debug.log
Row 171  Col 1   11:02  Ctrl-K H for help
recv_fe_driver_status>

recv_fe_interface_connect {''tx_shmem_frame'': 185025,
''rx_shmem_frame'':
185026, ''handle'': 0}

recv_fe_interface_connect {''tx_shmem_frame'': 185029,
''rx_shmem_frame'':
185030, ''handle'': 1}
{}
{}
VirqClient.virqReceived> 4
vif-bridge down vif=vif8.0 domain=niagras1 mac=02:00:00:01:02:00
bridge=xen-br0
vif-bridge down vif=vif8.1 domain=niagras1 mac=aa:00:00:53:7a:5b
bridge=xen-br0
VirqClient.virqReceived> 4
vif-bridge down vif=vif7.0 domain=vpn1dc1 mac=02:00:00:01:01:00
bridge=xen-br0
vif-bridge down vif=vif7.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
-bridge down vif=vif4.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
interface vif4.1 does not exist!
VIRTUAL MEMORY ARRANGEMENT:
 Loaded kernel: c0100000->c02c9df4
 Init. ramdisk: c02ca000->c02ca000
 Phys-Mach map: c02ca000->c02e2000
 Page tables:   c02e2000->c02e4000
 Start info:    c02e4000->c02e5000
 Boot stack:    c02e5000->c02e6000
 TOTAL:         c0000000->c0400000
 ENTRY ADDRESS: c0100000
vif-bridge up vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
bridge=xen-br0
interface vif5.0 does not exist!
vif-bridge up vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
interface vif5.1 does not exist!
recv_fe_driver_status> {''status'': 1}

recv_fe_driver_status>

recv_fe_interface_connect {''tx_shmem_frame'': 202459,
''rx_shmem_frame'':
202460, ''handle'': 0}

recv_fe_interface_connect {''tx_shmem_frame'': 202463,
''rx_shmem_frame'':
202464, ''handle'': 1}
VirqClient.virqReceived> 4
vif-bridge down vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
bridge=xen-br0
interface vif5.0 does not exist!
vif-bridge down vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
interface vif5.1 does not exist!

File xend-debug.log not changed so no update needed.
[root@xenpub1dc1 log]# ls
acpid            boot.log.1  cron.2   mail       messages    prelink.log
samba     spooler    wtmp            xfrd.log
anaconda.log     boot.log.2  cups     maillog    messages.1  rpmpkgs
secure    spooler.1  xend-debug.log  Xorg.0.log
anaconda.syslog  cron        dmesg    maillog.1  messages.2  rpmpkgs.1
secure.1  spooler.2  xend.log        Xorg.0.log.old
boot.log         cron.1      lastlog  maillog.2  ppp         rpmpkgs.2
secure.2  vbox       xend.log.1      yum.log
[root@xenpub1dc1 log]# joe xend.log
Processing ''/etc/joe/joerc''...done
Processing ''/etc/joe/joerc''...done









































    I    xend.log
Row 6079 Col 1   11:04  Ctrl-K H for help
[2006-01-28 22:06:09 xend] INFO (XendRoot:113) EVENT>
xend.domain.shutdown [''vpn1dc1'', ''2'',
''halt'']
[2006-01-28 22:06:09 xend] INFO (XendRoot:113) EVENT>
xend.domain.shutdown [''niagras1'', ''1'',
''halt'']
[2006-01-28 22:06:22 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=niagras1 id=1
[2006-01-28 22:06:22 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=1 reason=poweroff
[2006-01-28 22:06:22 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 1 poweroff 0
[2006-01-28 22:06:22 xend] INFO (XendDomain:568) Destroying domain:
name=niagras1
[2006-01-28 22:06:22 xend] DEBUG (XendDomainInfo:665) Destroying vifs
for domain 1
[2006-01-28 22:06:22 xend] DEBUG (netif:305) Destroying vif domain=1
vif=0
[2006-01-28 22:06:22 xend] DEBUG (netif:305) Destroying vif domain=1
vif=1
[2006-01-28 22:06:22 xend] DEBUG (XendDomainInfo:674) Destroying vbds
for domain 1
[2006-01-28 22:06:22 xend] DEBUG (blkif:552) Destroying blkif domain=1
[2006-01-28 22:06:22 xend] DEBUG (blkif:408) Destroying vbd domain=1
idx=0
[2006-01-28 22:06:22 xend] DEBUG (blkif:408) Destroying vbd domain=1
idx=1
[2006-01-28 22:06:22 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 1
[2006-01-28 22:06:22 xend] DEBUG (XendDomainInfo:622) Closing channel to
domain 1
[2006-01-28 22:06:22 xend] INFO (XendRoot:113) EVENT> xend.virq 4
[2006-01-28 22:06:22 xend] DEBUG (blkif:363) Unbinding vbd (type phy)
from VG1/niagras1root
[2006-01-28 22:06:22 xend] DEBUG (blkif:363) Unbinding vbd (type phy)
from VG1/niagras1swap
[2006-01-28 22:06:22 xend] INFO (console:58) Console disconnected 15
127.0.0.1 3484
[2006-01-28 22:06:22 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''niagras1'', ''1'',
''poweroff'']
[2006-01-28 22:06:22 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''niagras1'', ''1'']
[2006-01-28 22:06:22 xend] INFO (XendRoot:113) EVENT>
xend.console.disconnect [15, ''127.0.0.1'', 3484]
[2006-01-28 22:06:23 xend] INFO (XendRoot:113) EVENT> xend.domain.died
[''niagras1'', ''1'']
[2006-01-28 22:06:24 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:24 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:24 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:24 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:24 xend] DEBUG (XendDomainInfo:665) Destroying vifs
for domain 2
[2006-01-28 22:06:24 xend] DEBUG (netif:305) Destroying vif domain=2
vif=0
[2006-01-28 22:06:24 xend] DEBUG (netif:305) Destroying vif domain=2
vif=1
[2006-01-28 22:06:24 xend] DEBUG (XendDomainInfo:674) Destroying vbds
for domain 2
[2006-01-28 22:06:24 xend] DEBUG (blkif:552) Destroying blkif domain=2
[2006-01-28 22:06:24 xend] DEBUG (blkif:408) Destroying vbd domain=2
idx=0
[2006-01-28 22:06:24 xend] DEBUG (blkif:408) Destroying vbd domain=2
idx=1
[2006-01-28 22:06:24 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:24 xend] DEBUG (XendDomainInfo:622) Closing channel to
domain 2
[2006-01-28 22:06:24 xend] INFO (XendRoot:113) EVENT> xend.virq 4
[2006-01-28 22:06:24 xend] DEBUG (blkif:363) Unbinding vbd (type phy)
from VG1/vpn1dc1root
[2006-01-28 22:06:24 xend] DEBUG (blkif:363) Unbinding vbd (type phy)
from VG1/vpn1dc1swap
[2006-01-28 22:06:24 xend] INFO (console:58) Console disconnected 16
127.0.0.1 3986
[2006-01-28 22:06:24 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
Jan 28 22:05:49 xenpub1dc1 ntpd[3652]: ntpd exiting on signal 15
Jan 28 22:05:49 xenpub1dc1 ntpd: ntpd shutdown succeeded
Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.0 left promiscuous mode
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
disabled state
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
disabled state
Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.1 left promiscuous mode
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
disabled state
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
disabled state
Jan 28 22:06:23 xenpub1dc1 net.agent[4714]: remove event not handled
Jan 28 22:06:23 xenpub1dc1 net.agent[4723]: remove event not handled
Jan 28 22:06:23 xenpub1dc1 net.agent[4737]: remove event not handled
Jan 28 22:06:23 xenpub1dc1 net.agent[4746]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.0 left promiscuous mode
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.1 left promiscuous mode
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 net.agent[4774]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 net.agent[4766]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 net.agent[4789]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 net.agent[4798]: remove event not handled
Jan 28 22:18:03 xenpub1dc1 shutdown: shutting down for system reboot
Jan 28 22:18:03 xenpub1dc1 init: Switching to runlevel: 6
Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3781]: session closed for
user root
Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3779]: session closed for
user root
Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3780]: session closed for
user root
Jan 28 22:18:04 xenpub1dc1 xendomains: Shutting down all Xen domains:
Jan 28 22:18:05 xenpub1dc1 xendomains: Error: Internal Server Error
Jan 28 22:18:05 xenpub1dc1 xendomains:  failed
Jan 28 22:18:05 xenpub1dc1 xendomains:
Jan 28 22:18:05 xenpub1dc1 rc: Stopping xendomains:  failed
Jan 28 22:18:05 xenpub1dc1 kernel: Kernel logging (proc) stopped.
Jan 28 22:18:05 xenpub1dc1 kernel: Kernel log daemon terminating.
Jan 28 22:18:06 xenpub1dc1 syslog: klogd shutdown succeeded
Jan 28 22:18:06 xenpub1dc1 exiting on signal 15
Jan 28 22:20:06 xenpub1dc1 syslogd 1.4.1: restart.
Jan 28 22:20:06 xenpub1dc1 syslog: syslogd startup succeeded
Jan 28 22:20:06 xenpub1dc1 kernel: klogd 1.4.1, log source = /proc/kmsg
started.
Jan 28 22:20:06 xenpub1dc1 kernel: Linux version 2.6.11.12-xen0
(root@xenpub1dc1.atlasgroup.net) (gcc version 3.4.4 20050721 (Red Hat
3.4.4-2)) #4 Thu Jan
File messages not changed so no update needed.
[root@xenpub1dc1 log]# ls
acpid            boot.log.1  cron.2   mail       messages    prelink.log
samba     spooler    wtmp            xfrd.log
anaconda.log     boot.log.2  cups     maillog    messages.1  rpmpkgs
secure    spooler.1  xend-debug.log  Xorg.0.log
anaconda.syslog  cron        dmesg    maillog.1  messages.2  rpmpkgs.1
secure.1  spooler.2  xend.log        Xorg.0.log.old
boot.log         cron.1      lastlog  maillog.2  ppp         rpmpkgs.2
secure.2  vbox       xend.log.1      yum.log
[root@xenpub1dc1 log]# joe xend-debug.log
Processing ''/etc/joe/joerc''...done
Processing ''/etc/joe/joerc''...done









































    I    xend-debug.log
Row 171  Col 1   11:02  Ctrl-K H for help
recv_fe_driver_status>

recv_fe_interface_connect {''tx_shmem_frame'': 185025,
''rx_shmem_frame'':
185026, ''handle'': 0}

recv_fe_interface_connect {''tx_shmem_frame'': 185029,
''rx_shmem_frame'':
185030, ''handle'': 1}
{}
{}
VirqClient.virqReceived> 4
vif-bridge down vif=vif8.0 domain=niagras1 mac=02:00:00:01:02:00
bridge=xen-br0
vif-bridge down vif=vif8.1 domain=niagras1 mac=aa:00:00:53:7a:5b
bridge=xen-br0
VirqClient.virqReceived> 4
vif-bridge down vif=vif7.0 domain=vpn1dc1 mac=02:00:00:01:01:00
bridge=xen-br0
vif-bridge down vif=vif7.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
-bridge down vif=vif4.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
interface vif4.1 does not exist!
VIRTUAL MEMORY ARRANGEMENT:
 Loaded kernel: c0100000->c02c9df4
 Init. ramdisk: c02ca000->c02ca000
 Phys-Mach map: c02ca000->c02e2000
 Page tables:   c02e2000->c02e4000
 Start info:    c02e4000->c02e5000
 Boot stack:    c02e5000->c02e6000
 TOTAL:         c0000000->c0400000
 ENTRY ADDRESS: c0100000
vif-bridge up vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
bridge=xen-br0
interface vif5.0 does not exist!
vif-bridge up vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
interface vif5.1 does not exist!
recv_fe_driver_status> {''status'': 1}

recv_fe_driver_status>

recv_fe_interface_connect {''tx_shmem_frame'': 202459,
''rx_shmem_frame'':
202460, ''handle'': 0}

recv_fe_interface_connect {''tx_shmem_frame'': 202463,
''rx_shmem_frame'':
202464, ''handle'': 1}
VirqClient.virqReceived> 4
vif-bridge down vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
bridge=xen-br0
interface vif5.0 does not exist!
vif-bridge down vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
interface vif5.1 does not exist!

File xend-debug.log not changed so no update needed.
[root@xenpub1dc1 log]# ls
acpid            boot.log.1  cron.2   mail       messages    prelink.log
samba     spooler    wtmp            xfrd.log
anaconda.log     boot.log.2  cups     maillog    messages.1  rpmpkgs
secure    spooler.1  xend-debug.log  Xorg.0.log
anaconda.syslog  cron        dmesg    maillog.1  messages.2  rpmpkgs.1
secure.1  spooler.2  xend.log        Xorg.0.log.old
boot.log         cron.1      lastlog  maillog.2  ppp         rpmpkgs.2
secure.2  vbox       xend.log.1      yum.log
[root@xenpub1dc1 log]# joe xend.log
Processing ''/etc/joe/joerc''...done
Processing ''/etc/joe/joerc''...done









































    I    xend.log
Row 6077 Col 91  11:06  Ctrl-K H for help
[2006-01-28 22:06:24 xend] DEBUG (blkif:363) Unbinding vbd (type phy)
from VG1/vpn1dc1swap
[2006-01-28 22:06:24 xend] INFO (console:58) Console disconnected 16
127.0.0.1 3986
[2006-01-28 22:06:24 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:24 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:24 xend] INFO (XendRoot:113) EVENT>
xend.console.disconnect [16, ''127.0.0.1'', 3986]
[2006-01-28 22:06:26 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:26 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:26 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:26 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:26 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:26 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:26 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:28 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:28 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:28 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:28 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:28 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:28 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:28 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:30 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:30 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:30 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:30 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:30 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:30 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:30 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:32 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:32 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:32 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:32 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:32 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:32 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:32 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:34 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:34 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:34 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:34 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:34 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:34 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:34 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:36 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2  
Jan 28 22:05:49 xenpub1dc1 ntpd[3652]: ntpd exiting on signal 15
Jan 28 22:05:49 xenpub1dc1 ntpd: ntpd shutdown succeeded
Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.0 left promiscuous mode
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
disabled state
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
disabled state
Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.1 left promiscuous mode
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
disabled state
Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
disabled state
Jan 28 22:06:23 xenpub1dc1 net.agent[4714]: remove event not handled
Jan 28 22:06:23 xenpub1dc1 net.agent[4723]: remove event not handled
Jan 28 22:06:23 xenpub1dc1 net.agent[4737]: remove event not handled
Jan 28 22:06:23 xenpub1dc1 net.agent[4746]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.0 left promiscuous mode
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.1 left promiscuous mode
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
disabled state
Jan 28 22:06:24 xenpub1dc1 net.agent[4774]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 net.agent[4766]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 net.agent[4789]: remove event not handled
Jan 28 22:06:24 xenpub1dc1 net.agent[4798]: remove event not handled
Jan 28 22:18:03 xenpub1dc1 shutdown: shutting down for system reboot
Jan 28 22:18:03 xenpub1dc1 init: Switching to runlevel: 6
Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3781]: session closed for
user root
Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3779]: session closed for
user root
Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3780]: session closed for
user root
Jan 28 22:18:04 xenpub1dc1 xendomains: Shutting down all Xen domains:
Jan 28 22:18:05 xenpub1dc1 xendomains: Error: Internal Server Error
Jan 28 22:18:05 xenpub1dc1 xendomains:  failed
Jan 28 22:18:05 xenpub1dc1 xendomains:
Jan 28 22:18:05 xenpub1dc1 rc: Stopping xendomains:  failed
Jan 28 22:18:05 xenpub1dc1 kernel: Kernel logging (proc) stopped.
Jan 28 22:18:05 xenpub1dc1 kernel: Kernel log daemon terminating.
Jan 28 22:18:06 xenpub1dc1 syslog: klogd shutdown succeeded
Jan 28 22:18:06 xenpub1dc1 exiting on signal 15
Jan 28 22:20:06 xenpub1dc1 syslogd 1.4.1: restart.
Jan 28 22:20:06 xenpub1dc1 syslog: syslogd startup succeeded
Jan 28 22:20:06 xenpub1dc1 kernel: klogd 1.4.1, log source = /proc/kmsg
started.
Jan 28 22:20:06 xenpub1dc1 kernel: Linux version 2.6.11.12-xen0
(root@xenpub1dc1.atlasgroup.net) (gcc version 3.4.4 20050721 (Red Hat
3.4.4-2)) #4 Thu Jan
File messages not changed so no update needed.
[root@xenpub1dc1 log]# ls
acpid            boot.log.1  cron.2   mail       messages    prelink.log
samba     spooler    wtmp            xfrd.log
anaconda.log     boot.log.2  cups     maillog    messages.1  rpmpkgs
secure    spooler.1  xend-debug.log  Xorg.0.log
anaconda.syslog  cron        dmesg    maillog.1  messages.2  rpmpkgs.1
secure.1  spooler.2  xend.log        Xorg.0.log.old
boot.log         cron.1      lastlog  maillog.2  ppp         rpmpkgs.2
secure.2  vbox       xend.log.1      yum.log
[root@xenpub1dc1 log]# joe xend-debug.log
Processing ''/etc/joe/joerc''...done
Processing ''/etc/joe/joerc''...done









































    I    xend-debug.log
Row 171  Col 1   11:02  Ctrl-K H for help
recv_fe_driver_status>

recv_fe_interface_connect {''tx_shmem_frame'': 185025,
''rx_shmem_frame'':
185026, ''handle'': 0}

recv_fe_interface_connect {''tx_shmem_frame'': 185029,
''rx_shmem_frame'':
185030, ''handle'': 1}
{}
{}
VirqClient.virqReceived> 4
vif-bridge down vif=vif8.0 domain=niagras1 mac=02:00:00:01:02:00
bridge=xen-br0
vif-bridge down vif=vif8.1 domain=niagras1 mac=aa:00:00:53:7a:5b
bridge=xen-br0
VirqClient.virqReceived> 4
vif-bridge down vif=vif7.0 domain=vpn1dc1 mac=02:00:00:01:01:00
bridge=xen-br0
vif-bridge down vif=vif7.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
-bridge down vif=vif4.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
interface vif4.1 does not exist!
VIRTUAL MEMORY ARRANGEMENT:
 Loaded kernel: c0100000->c02c9df4
 Init. ramdisk: c02ca000->c02ca000
 Phys-Mach map: c02ca000->c02e2000
 Page tables:   c02e2000->c02e4000
 Start info:    c02e4000->c02e5000
 Boot stack:    c02e5000->c02e6000
 TOTAL:         c0000000->c0400000
 ENTRY ADDRESS: c0100000
vif-bridge up vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
bridge=xen-br0
interface vif5.0 does not exist!
vif-bridge up vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
interface vif5.1 does not exist!
recv_fe_driver_status> {''status'': 1}

recv_fe_driver_status>

recv_fe_interface_connect {''tx_shmem_frame'': 202459,
''rx_shmem_frame'':
202460, ''handle'': 0}

recv_fe_interface_connect {''tx_shmem_frame'': 202463,
''rx_shmem_frame'':
202464, ''handle'': 1}
VirqClient.virqReceived> 4
vif-bridge down vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
bridge=xen-br0
interface vif5.0 does not exist!
vif-bridge down vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
bridge=xen-br1
interface vif5.1 does not exist!

File xend-debug.log not changed so no update needed.
[root@xenpub1dc1 log]# ls
acpid            boot.log.1  cron.2   mail       messages    prelink.log
samba     spooler    wtmp            xfrd.log
anaconda.log     boot.log.2  cups     maillog    messages.1  rpmpkgs
secure    spooler.1  xend-debug.log  Xorg.0.log
anaconda.syslog  cron        dmesg    maillog.1  messages.2  rpmpkgs.1
secure.1  spooler.2  xend.log        Xorg.0.log.old
boot.log         cron.1      lastlog  maillog.2  ppp         rpmpkgs.2
secure.2  vbox       xend.log.1      yum.log
[root@xenpub1dc1 log]# joe xend.log
Processing ''/etc/joe/joerc''...done
Processing ''/etc/joe/joerc''...done









































    I    xend.log
Row 6157 Col 100 11:07  Ctrl-K H for help
[2006-01-28 22:06:36 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:36 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:36 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:36 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:36 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:36 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:36 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:38 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:38 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:38 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:38 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:38 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:38 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:38 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:40 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:40 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:40 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:40 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:40 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:40 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:40 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:42 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:42 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:42 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:42 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:42 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:42 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:42 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:44 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:44 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:44 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:44 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:44 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:44 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:44 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:46 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:46 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:46 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:46 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:46 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:46 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:46 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:48 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:48 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:48 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:48 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:48 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:48 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:48 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:50 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:50 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:50 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:50 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:50 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:50 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:50 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:52 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:52 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:52 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:52 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:52 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:52 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:52 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:54 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:54 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:54 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:54 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:54 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:54 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:54 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:56 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:56 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:56 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:56 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1
[2006-01-28 22:06:56 xend] DEBUG (XendDomainInfo:634) Closing console,
domain 2
[2006-01-28 22:06:56 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
[''vpn1dc1'', ''2'',
''poweroff'']
[2006-01-28 22:06:56 xend] INFO (XendRoot:113) EVENT>
xend.domain.destroy [''vpn1dc1'', ''2'']
[2006-01-28 22:06:57 xend] DEBUG (XendDomain:244) XendDomain>reap>
domain died name=vpn1dc1 id=2
[2006-01-28 22:06:57 xend] DEBUG (XendDomain:247) XendDomain>reap>
shutdown id=2 reason=poweroff
[2006-01-28 22:06:57 xend] DEBUG (XendDomain:487)
domain_restart_schedule> 2 poweroff 0
[2006-01-28 22:06:57 xend] INFO (XendDomain:568) Destroying domain:
name=vpn1dc1 
etc., etc., etc.

Here are the configuration files for the two domUs:
#  -*- mode: python; -*-
#===========================================================================#
Python configuration setup for ''xm create''.
# This script sets the parameters used when a domain is created using
''xm create''.
# You use a separate script for each domain you want to create, or 
# you can set the parameters for the domain on the xm command line.
#===========================================================================
#----------------------------------------------------------------------------
# Kernel image file.
kernel = "/boot/vmlinuz-2.4.30-xenUVPN"

# Optional ramdisk.
#ramdisk = "/boot/initrd.gz"

# The domain build function. Default is ''linux''.
#builder=''linux''

# Initial memory allocation (in megabytes) for the new domain.
memory = 96

# A name for your domain. All domains must have different names.
name = "niagras1"

# Which CPU to start domain on? 
#cpu = -1   # leave to Xen to pick

#----------------------------------------------------------------------------
# Define network interfaces.

# Number of network interfaces. Default is 1.
nics=2

# Optionally define mac and/or bridge for the network interfaces.
# Random MACs are assigned if not given.
#vif = [ ''mac=aa:00:00:00:00:11, bridge=xen-br0'' ]

vif = [''mac=02:00:00:01:02:00, bridge=xen-br0'' ]

#----------------------------------------------------------------------------
# Define the disk devices you want the domain to have access to, and
# what you want them accessible as.
# Each disk entry is of the form phy:UNAME,DEV,MODE
# where UNAME is the device, DEV is the device name the domain will see,
# and MODE is r for read-only, w for read-write.

#disk = [ ''phy:hda1,hda1,w'' ]

disk = [ ''phy:VG1/niagras1root,sda1,w'',
''phy:VG1/niagras1swap,sda2,w'' ]

#----------------------------------------------------------------------------
# Set the kernel command line for the new domain.
# You only need to define the IP parameters and hostname if the
domain''s
# IP config doesn''t, e.g. in ifcfg-eth0 or via DHCP.
# You can use ''extra'' to set the runlevel and custom
environment
# variables used by custom rc scripts (e.g. VMID=, usr= ).

# Set if you want dhcp to allocate the IP address.
#dhcp="dhcp"
# Set netmask.
#netmask# Set default gateway.
#gateway# Set the hostname.
#hostname= "vm%d" % vmid

# Set root device.
root = "/dev/sda1 ro"

# Root device for nfs.
#root = "/dev/nfs"
# The nfs server.
#nfs_server = ''169.254.1.0''  
# Root directory on the nfs server.
#nfs_root   = ''/full/path/to/root/directory''

# Sets runlevel 4.
#extra = "4"

#----------------------------------------------------------------------------
# Set according to whether you want the domain restarted when it exits.
# The default is ''onreboot'', which restarts the domain when it
shuts
down
# with exit code reboot.
# Other values are ''always'', and ''never''.

#restart = ''onreboot''

#===========================================================================

#  -*- mode: python; -*-
#===========================================================================#
Python configuration setup for ''xm create''.
# This script sets the parameters used when a domain is created using
''xm create''.
# You use a separate script for each domain you want to create, or 
# you can set the parameters for the domain on the xm command line.
#===========================================================================
#----------------------------------------------------------------------------
# Kernel image file.
kernel = "/boot/vmlinuz-2.4.30-xenUVPN"

# Optional ramdisk.
#ramdisk = "/boot/initrd.gz"

# The domain build function. Default is ''linux''.
#builder=''linux''

# Initial memory allocation (in megabytes) for the new domain.
memory = 96

# A name for your domain. All domains must have different names.
name = "vpn1dc1"

# Which CPU to start domain on? 
#cpu = -1   # leave to Xen to pick

#----------------------------------------------------------------------------
# Define network interfaces.

# Number of network interfaces. Default is 1.
nics=2

# Optionally define mac and/or bridge for the network interfaces.
# Random MACs are assigned if not given.
#vif = [ ''mac=aa:00:00:00:00:11, bridge=xen-br0'' ]

vif = [''mac=02:00:00:01:01:00, bridge=xen-br0'',
''mac=02:00:00:01:01:01,
bridge=xen-br1'' ]

#----------------------------------------------------------------------------
# Define the disk devices you want the domain to have access to, and
# what you want them accessible as.
# Each disk entry is of the form phy:UNAME,DEV,MODE
# where UNAME is the device, DEV is the device name the domain will see,
# and MODE is r for read-only, w for read-write.

#disk = [ ''phy:hda1,hda1,w'' ]

disk = [ ''phy:VG1/vpn1dc1root,sda1,w'',
''phy:VG1/vpn1dc1swap,sda2,w'' ]

#----------------------------------------------------------------------------
# Set the kernel command line for the new domain.
# You only need to define the IP parameters and hostname if the
domain''s
# IP config doesn''t, e.g. in ifcfg-eth0 or via DHCP.
# You can use ''extra'' to set the runlevel and custom
environment
# variables used by custom rc scripts (e.g. VMID=, usr= ).

# Set if you want dhcp to allocate the IP address.
#dhcp="dhcp"
# Set netmask.
#netmask# Set default gateway.
#gateway# Set the hostname.
#hostname= "vm%d" % vmid

# Set root device.
root = "/dev/sda1 ro"

# Root device for nfs.
#root = "/dev/nfs"
# The nfs server.
#nfs_server = ''169.254.1.0''  
# Root directory on the nfs server.
#nfs_root   = ''/full/path/to/root/directory''

# Sets runlevel 4.
#extra = "4"

#----------------------------------------------------------------------------
# Set according to whether you want the domain restarted when it exits.
# The default is ''onreboot'', which restarts the domain when it
shuts
down
# with exit code reboot.
# Other values are ''always'', and ''never''.

#restart = ''onreboot''

#===========================================================================

I''m stumped but it sure looks serious.  Have I produced some dreadful
misconfiguration? This host is being shipped to a remote data center on
Monday and goes into production in a week.  Any help would be greatly
appreciated.  Thanks - John
-- 
John A. Sullivan III
Open Source Development Corporation
+1 207-985-7880
jsullivan@opensourcedevel.com

Financially sustainable open source development
http://www.opensourcedevel.com


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

More info on the network script at the bottom - John

On Sat, 2006-01-28 at 23:14 -0500, John A. Sullivan III
wrote:
> Pardon me for a long detailed post but this looks pretty serious.  The
> bottom line is that, when I attempt to shut down or reboot my simple
> xen-2.0.7 server running FC3 with two domUs, shutting down xendomains
> appears to send python into an infinite loop.  This may be due to a
> serious misconfiguration on my part but it is certainly a big problem.
> 
> If we shutdown the domUs manually first, the system reboots perfectly
> fine.
> 
> I''ll describe some history and include lots of log information. 
The
> configuration is slightly unusual.  Both domUs are VPN gateways running
> the 2.4.30 kernel with openswan and openvpn.  Because of the openvpn,
> but are privileged domains but the only device driver configured in the
> kernel is the openvpn tun/tap adapter.
> 
> We first noticed problems during shutdown in that the last domU to
> shutdown crashed very nastily when closing openvpn.  We could see this
> even when manually stopping the domUs.  We surmised that the problem was
> we had tun0 and tun1 defined in both domains and killing them in one
> must be killing them in another.  We renamed those tun adapters in the
> second domain to tun10 and tun11 and that problem went away.
> 
> We still noticed very long reboot times.  The hang seemed to be at
> killall but the "logjam" seemed to break after xendomains errored
out.
> We assumed this was because killall killed something that xendomains
> needed to shutdown gracefully so we added explicit K scripts in rc0.d
> and rc6.d for xend and xendomains.
> 
> At first this seemed to work but it must have been only coincidental
> because the problem returned.  We then realized we had brain cramped and
> shutdown xend before xendomains.  We thought for sure we had found the
> problem and put them in the proper order but, alas, the problem did not
> go away.
> 
> To gain a little more insight into exactly what was happening, we
> manually ran each kill script in rc6.d.  When we ran K78xendomains stop,
> we saw the problem, or at least many aspect of it.
> 
> >From a xen console, we saw that the first domU had cleanly shutdown but
> the second one was still listed howbeit with zero memory.
> 
> The hard disks were thrashing madly and top showed python consuming 99.9
> percent CPU utilization.  Most of the time, during reboot, we do not see
> the madly thrashing disks.  They do so for a moment as one might expect
> when shutting down the domains but then just blip occasionally
> thereafter.
> 
> Here is what we see in the logs:
> 
> >From messages (ntpd was the last service stopped before xendomains):
> Jan 28 22:05:49 xenpub1dc1 ntpd[3652]: ntpd exiting on signal 15
> Jan 28 22:05:49 xenpub1dc1 ntpd: ntpd shutdown succeeded
> Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.0 left promiscuous mode
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
> disabled state
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
> disabled state
> Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.1 left promiscuous mode
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
> disabled state
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
> disabled state
> Jan 28 22:06:23 xenpub1dc1 net.agent[4714]: remove event not handled
> Jan 28 22:06:23 xenpub1dc1 net.agent[4723]: remove event not handled
> Jan 28 22:06:23 xenpub1dc1 net.agent[4737]: remove event not handled
> Jan 28 22:06:23 xenpub1dc1 net.agent[4746]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.0 left promiscuous mode
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.1 left promiscuous mode
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 net.agent[4774]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 net.agent[4766]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 net.agent[4789]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 net.agent[4798]: remove event not handled
> 
> Here is the complete xend-debug.log:
> network start bridge=xen-br0 netdev=eth0 antispoof=no
> VIRTUAL MEMORY ARRANGEMENT:
>  Loaded kernel: c0100000->c0319394
>  Init. ramdisk: c031a000->c031a000
>  Phys-Mach map: c031a000->c0332000
>  Page tables:   c0332000->c0334000
>  Start info:    c0334000->c0335000
>  Boot stack:    c0335000->c0336000
>  TOTAL:         c0000000->c0400000
>  ENTRY ADDRESS: c0100000
> vif-bridge up vif=vif1.0 domain=niagras1 mac=02:00:00:01:02:00
> bridge=xen-br0
> vif-bridge up vif=vif1.1 domain=niagras1 mac=aa:00:00:58:4a:a7
> bridge=xen-br0
> VIRTUAL MEMORY ARRANGEMENT:
>  Loaded kernel: c0100000->c0319394
>  Init. ramdisk: c031a000->c031a000
>  Phys-Mach map: c031a000->c0332000
>  Page tables:   c0332000->c0334000
>  Start info:    c0334000->c0335000
>  Boot stack:    c0335000->c0336000
>  TOTAL:         c0000000->c0400000
>  ENTRY ADDRESS: c0100000
> vif-bridge up vif=vif2.0 domain=vpn1dc1 mac=02:00:00:01:01:00
> bridge=xen-br0
> vif-bridge up vif=vif2.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> recv_fe_driver_status> {''status'': 1}
> 
> recv_fe_driver_status>
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 44763,
''rx_shmem_frame'':
> 44764, ''handle'': 0}
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 44767,
''rx_shmem_frame'':
> 44768, ''handle'': 1}
> recv_fe_driver_status> {''status'': 1}
> 
> recv_fe_driver_status>
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 69339,
''rx_shmem_frame'':
> 69340, ''handle'': 0}
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 69343,
''rx_shmem_frame'':
> 69344, ''handle'': 1}
> {}
> {}
> VirqClient.virqReceived> 4
> vif-bridge down vif=vif2.0 domain=vpn1dc1 mac=02:00:00:01:01:00
> bridge=xen-br0
> vif-bridge down vif=vif2.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> VirqClient.virqReceived> 4
> vif-bridge down vif=vif1.0 domain=niagras1 mac=02:00:00:01:02:00
> bridge=xen-br0
> vif-bridge down vif=vif1.1 domain=niagras1 mac=aa:00:00:26:24:c4
> bridge=xen-br0
> Traceback (most recent call last):
>   File
"/usr/lib/python2.3/site-packages/twisted/protocols/basic.py",
> line 229, in dataReceived
>     return self.rawDataReceived(data)
>   File
"/usr/lib/python2.3/site-packages/twisted/protocols/http.py",
> line 1015, in rawDataReceived
>     self.allContentReceived()
>   File
"/usr/lib/python2.3/site-packages/twisted/protocols/http.py",
> line 1006, in allContentReceived
>     req.requestReceived(command, path, version)
>   File
"/usr/lib/python2.3/site-packages/twisted/protocols/http.py",
> line 557, in requestReceived
>     self.process()
> --- <exception caught here> ---
>   File "/usr/lib/python2.3/site-packages/twisted/web/server.py",
line
> 165, in process
>     self.render(resrc)
>   File "/usr/lib/python2.3/site-packages/twisted/web/server.py",
line
> 172, in render
>     body = resrc.render(self)
>   File
"/usr/lib/python2.3/site-packages/twisted/web/resource.py", line
> 201, in render
>     return m(request)
>   File "/usr/lib/python/xen/xend/server/SrvDomain.py", line 180,
in
> render_POST
>     return self.perform(req)
>   File "/usr/lib/python/xen/xend/server/SrvBase.py", line 103, in
> perform
>     return self._perform(op, op_method, req)
>   File "/usr/lib/python/xen/xend/server/SrvBase.py", line 109, in
> _perform
>     return self._perform_err(err, op, req)
>   File "/usr/lib/python/xen/xend/server/SrvBase.py", line 107, in
> _perform
>     val = op_method(op, req)
>   File "/usr/lib/python/xen/xend/server/SrvDomain.py", line 47,
in
> op_shutdown
>     val = fn(req.args, {''dom'': self.dom.id})
>   File "/usr/lib/python/xen/xend/Args.py", line 145, in __call__
>     return self.call_with_form_args(self.fn, fargs, xargs=xargs)
>   File "/usr/lib/python/xen/xend/Args.py", line 121, in
> call_with_form_args
>     return fn(*params, **keys)
>   File "/usr/lib/python/xen/xend/XendDomain.py", line 477, in
> domain_shutdown
>     val = xend.domain_shutdown(dominfo.id, reason, key)
>   File "/usr/lib/python/xen/xend/server/SrvDaemon.py", line 723,
in
> domain_shutdown
>     ctrl = self.domainCF.getController(dom)
>   File "/usr/lib/python/xen/xend/server/controller.py", line 323,
in
> getController
>     ctrl = self.createController(dom)
>   File "/usr/lib/python/xen/xend/server/domain.py", line 20, in
> createController
>     return DomainController(self, dom)
>   File "/usr/lib/python/xen/xend/server/domain.py", line 38, in
__init__
>     self.registerChannel()
>   File "/usr/lib/python/xen/xend/server/controller.py", line 220,
in
> registerChannel
>     self.channel = self.channelFactory.domChannel(self.dom)
>   File "/usr/lib/python/xen/xend/server/channel.py", line 83, in
> domChannel
>     remote_port=remote_port)
>   File "/usr/lib/python/xen/xend/server/channel.py", line 253, in
> __init__
>     remote_port=remote_port)
>   File "/usr/lib/python/xen/xend/server/channel.py", line 137, in
> createPort
>     remote_port=int(remote_port))
> xen.lowlevel.xu.PortError: Failed to map domain control interface
> /usr/lib/python2.3/site-packages/twisted/web/util.py:290: FutureWarning:
> hex()/oct() of negative int will return a signed string in Python 2.4
> and up
>   return ''''''<div
class="instance"><span class="instanceName">%s
instance
> @ %s</span>
> /usr/lib/python2.3/site-packages/twisted/web/util.py:266: FutureWarning:
> hex()/oct() of negative int will return a signed string in Python 2.4
> and up
>   w(''<div class="dict"><span
class="heading">Dictionary instance @ %
> s</span>'' % hex(id(d)))
> /usr/lib/python2.3/site-packages/twisted/web/util.py:279: FutureWarning:
> hex()/oct() of negative int will return a signed string in Python 2.4
> and up
>   w(''<div class="list"><span
class="heading">List instance @ %s</span>''
> % hex(id(l)))
> 
>  Init. ramdisk: c031a000->c031a000
>  Phys-Mach map: c031a000->c0332000
>  Page tables:   c0332000->c0334000
>  Start info:    c0334000->c0335000
>  Boot stack:    c0335000->c0336000
>  TOTAL:         c0000000->c0400000
>  ENTRY ADDRESS: c0100000
> vif-bridge up vif=vif7.0 domain=vpn1dc1 mac=02:00:00:01:01:00
> bridge=xen-br0
> vif-bridge up vif=vif7.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> recv_fe_driver_status> {''status'': 1}
> 
> recv_fe_driver_status>
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 88257,
''rx_shmem_frame'':
> 88258, ''handle'': 0}
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 88261,
''rx_shmem_frame'':
> 88262, ''handle'': 1}
> VIRTUAL MEMORY ARRANGEMENT:
>  Loaded kernel: c0100000->c0319394
>  Init. ramdisk: c031a000->c031a000
>  Phys-Mach map: c031a000->c0332000
>  Page tables:   c0332000->c0334000
>  Start info:    c0334000->c0335000
>  Boot stack:    c0335000->c0336000
>  TOTAL:         c0000000->c0400000
>  ENTRY ADDRESS: c0100000
> vif-bridge up vif=vif8.0 domain=niagras1 mac=02:00:00:01:02:00
> bridge=xen-br0
> vif-bridge up vif=vif8.1 domain=niagras1 mac=aa:00:00:53:7a:5b
> bridge=xen-br0
> recv_fe_driver_status> {''status'': 1}
> 
> recv_fe_driver_status>
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 185025,
''rx_shmem_frame'':
> 185026, ''handle'': 0}
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 185029,
''rx_shmem_frame'':
> 185030, ''handle'': 1}
> {}
> {}
> VirqClient.virqReceived> 4
> vif-bridge down vif=vif8.0 domain=niagras1 mac=02:00:00:01:02:00
> bridge=xen-br0
> vif-bridge down vif=vif8.1 domain=niagras1 mac=aa:00:00:53:7a:5b
> bridge=xen-br0
> VirqClient.virqReceived> 4
> vif-bridge down vif=vif7.0 domain=vpn1dc1 mac=02:00:00:01:01:00
> bridge=xen-br0
> vif-bridge down vif=vif7.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> -bridge down vif=vif4.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> interface vif4.1 does not exist!
> VIRTUAL MEMORY ARRANGEMENT:
>  Loaded kernel: c0100000->c02c9df4
>  Init. ramdisk: c02ca000->c02ca000
>  Phys-Mach map: c02ca000->c02e2000
>  Page tables:   c02e2000->c02e4000
>  Start info:    c02e4000->c02e5000
>  Boot stack:    c02e5000->c02e6000
>  TOTAL:         c0000000->c0400000
>  ENTRY ADDRESS: c0100000
> vif-bridge up vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
> bridge=xen-br0
> interface vif5.0 does not exist!
> vif-bridge up vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> interface vif5.1 does not exist!
> recv_fe_driver_status> {''status'': 1}
> 
> recv_fe_driver_status>
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 202459,
''rx_shmem_frame'':
> 202460, ''handle'': 0}
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 202463,
''rx_shmem_frame'':
> 202464, ''handle'': 1}
> VirqClient.virqReceived> 4
> vif-bridge down vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
> bridge=xen-br0
> interface vif5.0 does not exist!
> vif-bridge down vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> interface vif5.1 does not exist!
> 
> Here is the information from xend.log:
> Jan 28 22:05:49 xenpub1dc1 ntpd[3652]: ntpd exiting on signal 15
> Jan 28 22:05:49 xenpub1dc1 ntpd: ntpd shutdown succeeded
> Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.0 left promiscuous mode
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
> disabled state
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
> disabled state
> Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.1 left promiscuous mode
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
> disabled state
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
> disabled state
> Jan 28 22:06:23 xenpub1dc1 net.agent[4714]: remove event not handled
> Jan 28 22:06:23 xenpub1dc1 net.agent[4723]: remove event not handled
> Jan 28 22:06:23 xenpub1dc1 net.agent[4737]: remove event not handled
> Jan 28 22:06:23 xenpub1dc1 net.agent[4746]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.0 left promiscuous mode
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.1 left promiscuous mode
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 net.agent[4774]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 net.agent[4766]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 net.agent[4789]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 net.agent[4798]: remove event not handled
> Jan 28 22:18:03 xenpub1dc1 shutdown: shutting down for system reboot
> Jan 28 22:18:03 xenpub1dc1 init: Switching to runlevel: 6
> Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3781]: session closed for
> user root
> Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3779]: session closed for
> user root
> Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3780]: session closed for
> user root
> Jan 28 22:18:04 xenpub1dc1 xendomains: Shutting down all Xen domains:
> Jan 28 22:18:05 xenpub1dc1 xendomains: Error: Internal Server Error
> Jan 28 22:18:05 xenpub1dc1 xendomains:  failed
> Jan 28 22:18:05 xenpub1dc1 xendomains:
> Jan 28 22:18:05 xenpub1dc1 rc: Stopping xendomains:  failed
> Jan 28 22:18:05 xenpub1dc1 kernel: Kernel logging (proc) stopped.
> Jan 28 22:18:05 xenpub1dc1 kernel: Kernel log daemon terminating.
> Jan 28 22:18:06 xenpub1dc1 syslog: klogd shutdown succeeded
> Jan 28 22:18:06 xenpub1dc1 exiting on signal 15
> Jan 28 22:20:06 xenpub1dc1 syslogd 1.4.1: restart.
> Jan 28 22:20:06 xenpub1dc1 syslog: syslogd startup succeeded
> Jan 28 22:20:06 xenpub1dc1 kernel: klogd 1.4.1, log source = /proc/kmsg
> started.
> Jan 28 22:20:06 xenpub1dc1 kernel: Linux version 2.6.11.12-xen0
> (root@xenpub1dc1.atlasgroup.net) (gcc version 3.4.4 20050721 (Red Hat
> 3.4.4-2)) #4 Thu Jan
> File messages not changed so no update needed.
> [root@xenpub1dc1 log]# ls
> acpid            boot.log.1  cron.2   mail       messages    prelink.log
> samba     spooler    wtmp            xfrd.log
> anaconda.log     boot.log.2  cups     maillog    messages.1  rpmpkgs
> secure    spooler.1  xend-debug.log  Xorg.0.log
> anaconda.syslog  cron        dmesg    maillog.1  messages.2  rpmpkgs.1
> secure.1  spooler.2  xend.log        Xorg.0.log.old
> boot.log         cron.1      lastlog  maillog.2  ppp         rpmpkgs.2
> secure.2  vbox       xend.log.1      yum.log
> [root@xenpub1dc1 log]# joe xend-debug.log
> Processing ''/etc/joe/joerc''...done
> Processing ''/etc/joe/joerc''...done
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>     I    xend-debug.log
> Row 171  Col 1   11:02  Ctrl-K H for help
> recv_fe_driver_status>
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 185025,
''rx_shmem_frame'':
> 185026, ''handle'': 0}
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 185029,
''rx_shmem_frame'':
> 185030, ''handle'': 1}
> {}
> {}
> VirqClient.virqReceived> 4
> vif-bridge down vif=vif8.0 domain=niagras1 mac=02:00:00:01:02:00
> bridge=xen-br0
> vif-bridge down vif=vif8.1 domain=niagras1 mac=aa:00:00:53:7a:5b
> bridge=xen-br0
> VirqClient.virqReceived> 4
> vif-bridge down vif=vif7.0 domain=vpn1dc1 mac=02:00:00:01:01:00
> bridge=xen-br0
> vif-bridge down vif=vif7.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> -bridge down vif=vif4.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> interface vif4.1 does not exist!
> VIRTUAL MEMORY ARRANGEMENT:
>  Loaded kernel: c0100000->c02c9df4
>  Init. ramdisk: c02ca000->c02ca000
>  Phys-Mach map: c02ca000->c02e2000
>  Page tables:   c02e2000->c02e4000
>  Start info:    c02e4000->c02e5000
>  Boot stack:    c02e5000->c02e6000
>  TOTAL:         c0000000->c0400000
>  ENTRY ADDRESS: c0100000
> vif-bridge up vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
> bridge=xen-br0
> interface vif5.0 does not exist!
> vif-bridge up vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> interface vif5.1 does not exist!
> recv_fe_driver_status> {''status'': 1}
> 
> recv_fe_driver_status>
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 202459,
''rx_shmem_frame'':
> 202460, ''handle'': 0}
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 202463,
''rx_shmem_frame'':
> 202464, ''handle'': 1}
> VirqClient.virqReceived> 4
> vif-bridge down vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
> bridge=xen-br0
> interface vif5.0 does not exist!
> vif-bridge down vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> interface vif5.1 does not exist!
> 
> File xend-debug.log not changed so no update needed.
> [root@xenpub1dc1 log]# ls
> acpid            boot.log.1  cron.2   mail       messages    prelink.log
> samba     spooler    wtmp            xfrd.log
> anaconda.log     boot.log.2  cups     maillog    messages.1  rpmpkgs
> secure    spooler.1  xend-debug.log  Xorg.0.log
> anaconda.syslog  cron        dmesg    maillog.1  messages.2  rpmpkgs.1
> secure.1  spooler.2  xend.log        Xorg.0.log.old
> boot.log         cron.1      lastlog  maillog.2  ppp         rpmpkgs.2
> secure.2  vbox       xend.log.1      yum.log
> [root@xenpub1dc1 log]# joe xend.log
> Processing ''/etc/joe/joerc''...done
> Processing ''/etc/joe/joerc''...done
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>     I    xend.log
> Row 6079 Col 1   11:04  Ctrl-K H for help
> [2006-01-28 22:06:09 xend] INFO (XendRoot:113) EVENT>
> xend.domain.shutdown [''vpn1dc1'', ''2'',
''halt'']
> [2006-01-28 22:06:09 xend] INFO (XendRoot:113) EVENT>
> xend.domain.shutdown [''niagras1'', ''1'',
''halt'']
> [2006-01-28 22:06:22 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=niagras1 id=1
> [2006-01-28 22:06:22 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=1 reason=poweroff
> [2006-01-28 22:06:22 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 1 poweroff 0
> [2006-01-28 22:06:22 xend] INFO (XendDomain:568) Destroying domain:
> name=niagras1
> [2006-01-28 22:06:22 xend] DEBUG (XendDomainInfo:665) Destroying vifs
> for domain 1
> [2006-01-28 22:06:22 xend] DEBUG (netif:305) Destroying vif domain=1
> vif=0
> [2006-01-28 22:06:22 xend] DEBUG (netif:305) Destroying vif domain=1
> vif=1
> [2006-01-28 22:06:22 xend] DEBUG (XendDomainInfo:674) Destroying vbds
> for domain 1
> [2006-01-28 22:06:22 xend] DEBUG (blkif:552) Destroying blkif domain=1
> [2006-01-28 22:06:22 xend] DEBUG (blkif:408) Destroying vbd domain=1
> idx=0
> [2006-01-28 22:06:22 xend] DEBUG (blkif:408) Destroying vbd domain=1
> idx=1
> [2006-01-28 22:06:22 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 1
> [2006-01-28 22:06:22 xend] DEBUG (XendDomainInfo:622) Closing channel to
> domain 1
> [2006-01-28 22:06:22 xend] INFO (XendRoot:113) EVENT> xend.virq 4
> [2006-01-28 22:06:22 xend] DEBUG (blkif:363) Unbinding vbd (type phy)
> from VG1/niagras1root
> [2006-01-28 22:06:22 xend] DEBUG (blkif:363) Unbinding vbd (type phy)
> from VG1/niagras1swap
> [2006-01-28 22:06:22 xend] INFO (console:58) Console disconnected 15
> 127.0.0.1 3484
> [2006-01-28 22:06:22 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''niagras1'', ''1'',
''poweroff'']
> [2006-01-28 22:06:22 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''niagras1'', ''1'']
> [2006-01-28 22:06:22 xend] INFO (XendRoot:113) EVENT>
> xend.console.disconnect [15, ''127.0.0.1'', 3484]
> [2006-01-28 22:06:23 xend] INFO (XendRoot:113) EVENT> xend.domain.died
> [''niagras1'', ''1'']
> [2006-01-28 22:06:24 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:24 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:24 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:24 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:24 xend] DEBUG (XendDomainInfo:665) Destroying vifs
> for domain 2
> [2006-01-28 22:06:24 xend] DEBUG (netif:305) Destroying vif domain=2
> vif=0
> [2006-01-28 22:06:24 xend] DEBUG (netif:305) Destroying vif domain=2
> vif=1
> [2006-01-28 22:06:24 xend] DEBUG (XendDomainInfo:674) Destroying vbds
> for domain 2
> [2006-01-28 22:06:24 xend] DEBUG (blkif:552) Destroying blkif domain=2
> [2006-01-28 22:06:24 xend] DEBUG (blkif:408) Destroying vbd domain=2
> idx=0
> [2006-01-28 22:06:24 xend] DEBUG (blkif:408) Destroying vbd domain=2
> idx=1
> [2006-01-28 22:06:24 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:24 xend] DEBUG (XendDomainInfo:622) Closing channel to
> domain 2
> [2006-01-28 22:06:24 xend] INFO (XendRoot:113) EVENT> xend.virq 4
> [2006-01-28 22:06:24 xend] DEBUG (blkif:363) Unbinding vbd (type phy)
> from VG1/vpn1dc1root
> [2006-01-28 22:06:24 xend] DEBUG (blkif:363) Unbinding vbd (type phy)
> from VG1/vpn1dc1swap
> [2006-01-28 22:06:24 xend] INFO (console:58) Console disconnected 16
> 127.0.0.1 3986
> [2006-01-28 22:06:24 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> Jan 28 22:05:49 xenpub1dc1 ntpd[3652]: ntpd exiting on signal 15
> Jan 28 22:05:49 xenpub1dc1 ntpd: ntpd shutdown succeeded
> Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.0 left promiscuous mode
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
> disabled state
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
> disabled state
> Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.1 left promiscuous mode
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
> disabled state
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
> disabled state
> Jan 28 22:06:23 xenpub1dc1 net.agent[4714]: remove event not handled
> Jan 28 22:06:23 xenpub1dc1 net.agent[4723]: remove event not handled
> Jan 28 22:06:23 xenpub1dc1 net.agent[4737]: remove event not handled
> Jan 28 22:06:23 xenpub1dc1 net.agent[4746]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.0 left promiscuous mode
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.1 left promiscuous mode
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 net.agent[4774]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 net.agent[4766]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 net.agent[4789]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 net.agent[4798]: remove event not handled
> Jan 28 22:18:03 xenpub1dc1 shutdown: shutting down for system reboot
> Jan 28 22:18:03 xenpub1dc1 init: Switching to runlevel: 6
> Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3781]: session closed for
> user root
> Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3779]: session closed for
> user root
> Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3780]: session closed for
> user root
> Jan 28 22:18:04 xenpub1dc1 xendomains: Shutting down all Xen domains:
> Jan 28 22:18:05 xenpub1dc1 xendomains: Error: Internal Server Error
> Jan 28 22:18:05 xenpub1dc1 xendomains:  failed
> Jan 28 22:18:05 xenpub1dc1 xendomains:
> Jan 28 22:18:05 xenpub1dc1 rc: Stopping xendomains:  failed
> Jan 28 22:18:05 xenpub1dc1 kernel: Kernel logging (proc) stopped.
> Jan 28 22:18:05 xenpub1dc1 kernel: Kernel log daemon terminating.
> Jan 28 22:18:06 xenpub1dc1 syslog: klogd shutdown succeeded
> Jan 28 22:18:06 xenpub1dc1 exiting on signal 15
> Jan 28 22:20:06 xenpub1dc1 syslogd 1.4.1: restart.
> Jan 28 22:20:06 xenpub1dc1 syslog: syslogd startup succeeded
> Jan 28 22:20:06 xenpub1dc1 kernel: klogd 1.4.1, log source = /proc/kmsg
> started.
> Jan 28 22:20:06 xenpub1dc1 kernel: Linux version 2.6.11.12-xen0
> (root@xenpub1dc1.atlasgroup.net) (gcc version 3.4.4 20050721 (Red Hat
> 3.4.4-2)) #4 Thu Jan
> File messages not changed so no update needed.
> [root@xenpub1dc1 log]# ls
> acpid            boot.log.1  cron.2   mail       messages    prelink.log
> samba     spooler    wtmp            xfrd.log
> anaconda.log     boot.log.2  cups     maillog    messages.1  rpmpkgs
> secure    spooler.1  xend-debug.log  Xorg.0.log
> anaconda.syslog  cron        dmesg    maillog.1  messages.2  rpmpkgs.1
> secure.1  spooler.2  xend.log        Xorg.0.log.old
> boot.log         cron.1      lastlog  maillog.2  ppp         rpmpkgs.2
> secure.2  vbox       xend.log.1      yum.log
> [root@xenpub1dc1 log]# joe xend-debug.log
> Processing ''/etc/joe/joerc''...done
> Processing ''/etc/joe/joerc''...done
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>     I    xend-debug.log
> Row 171  Col 1   11:02  Ctrl-K H for help
> recv_fe_driver_status>
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 185025,
''rx_shmem_frame'':
> 185026, ''handle'': 0}
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 185029,
''rx_shmem_frame'':
> 185030, ''handle'': 1}
> {}
> {}
> VirqClient.virqReceived> 4
> vif-bridge down vif=vif8.0 domain=niagras1 mac=02:00:00:01:02:00
> bridge=xen-br0
> vif-bridge down vif=vif8.1 domain=niagras1 mac=aa:00:00:53:7a:5b
> bridge=xen-br0
> VirqClient.virqReceived> 4
> vif-bridge down vif=vif7.0 domain=vpn1dc1 mac=02:00:00:01:01:00
> bridge=xen-br0
> vif-bridge down vif=vif7.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> -bridge down vif=vif4.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> interface vif4.1 does not exist!
> VIRTUAL MEMORY ARRANGEMENT:
>  Loaded kernel: c0100000->c02c9df4
>  Init. ramdisk: c02ca000->c02ca000
>  Phys-Mach map: c02ca000->c02e2000
>  Page tables:   c02e2000->c02e4000
>  Start info:    c02e4000->c02e5000
>  Boot stack:    c02e5000->c02e6000
>  TOTAL:         c0000000->c0400000
>  ENTRY ADDRESS: c0100000
> vif-bridge up vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
> bridge=xen-br0
> interface vif5.0 does not exist!
> vif-bridge up vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> interface vif5.1 does not exist!
> recv_fe_driver_status> {''status'': 1}
> 
> recv_fe_driver_status>
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 202459,
''rx_shmem_frame'':
> 202460, ''handle'': 0}
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 202463,
''rx_shmem_frame'':
> 202464, ''handle'': 1}
> VirqClient.virqReceived> 4
> vif-bridge down vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
> bridge=xen-br0
> interface vif5.0 does not exist!
> vif-bridge down vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> interface vif5.1 does not exist!
> 
> File xend-debug.log not changed so no update needed.
> [root@xenpub1dc1 log]# ls
> acpid            boot.log.1  cron.2   mail       messages    prelink.log
> samba     spooler    wtmp            xfrd.log
> anaconda.log     boot.log.2  cups     maillog    messages.1  rpmpkgs
> secure    spooler.1  xend-debug.log  Xorg.0.log
> anaconda.syslog  cron        dmesg    maillog.1  messages.2  rpmpkgs.1
> secure.1  spooler.2  xend.log        Xorg.0.log.old
> boot.log         cron.1      lastlog  maillog.2  ppp         rpmpkgs.2
> secure.2  vbox       xend.log.1      yum.log
> [root@xenpub1dc1 log]# joe xend.log
> Processing ''/etc/joe/joerc''...done
> Processing ''/etc/joe/joerc''...done
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>     I    xend.log
> Row 6077 Col 91  11:06  Ctrl-K H for help
> [2006-01-28 22:06:24 xend] DEBUG (blkif:363) Unbinding vbd (type phy)
> from VG1/vpn1dc1swap
> [2006-01-28 22:06:24 xend] INFO (console:58) Console disconnected 16
> 127.0.0.1 3986
> [2006-01-28 22:06:24 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:24 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:24 xend] INFO (XendRoot:113) EVENT>
> xend.console.disconnect [16, ''127.0.0.1'', 3986]
> [2006-01-28 22:06:26 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:26 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:26 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:26 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:26 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:26 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:26 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:28 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:28 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:28 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:28 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:28 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:28 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:28 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:30 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:30 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:30 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:30 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:30 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:30 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:30 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:32 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:32 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:32 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:32 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:32 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:32 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:32 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:34 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:34 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:34 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:34 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:34 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:34 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:34 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:36 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2  
> Jan 28 22:05:49 xenpub1dc1 ntpd[3652]: ntpd exiting on signal 15
> Jan 28 22:05:49 xenpub1dc1 ntpd: ntpd shutdown succeeded
> Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.0 left promiscuous mode
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
> disabled state
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 2(vif1.0) entering
> disabled state
> Jan 28 22:06:22 xenpub1dc1 kernel: device vif1.1 left promiscuous mode
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
> disabled state
> Jan 28 22:06:22 xenpub1dc1 kernel: xen-br0: port 3(vif1.1) entering
> disabled state
> Jan 28 22:06:23 xenpub1dc1 net.agent[4714]: remove event not handled
> Jan 28 22:06:23 xenpub1dc1 net.agent[4723]: remove event not handled
> Jan 28 22:06:23 xenpub1dc1 net.agent[4737]: remove event not handled
> Jan 28 22:06:23 xenpub1dc1 net.agent[4746]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.0 left promiscuous mode
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br0: port 4(vif2.0) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 kernel: device vif2.1 left promiscuous mode
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 kernel: xen-br1: port 2(vif2.1) entering
> disabled state
> Jan 28 22:06:24 xenpub1dc1 net.agent[4774]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 net.agent[4766]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 net.agent[4789]: remove event not handled
> Jan 28 22:06:24 xenpub1dc1 net.agent[4798]: remove event not handled
> Jan 28 22:18:03 xenpub1dc1 shutdown: shutting down for system reboot
> Jan 28 22:18:03 xenpub1dc1 init: Switching to runlevel: 6
> Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3781]: session closed for
> user root
> Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3779]: session closed for
> user root
> Jan 28 22:18:03 xenpub1dc1 login(pam_unix)[3780]: session closed for
> user root
> Jan 28 22:18:04 xenpub1dc1 xendomains: Shutting down all Xen domains:
> Jan 28 22:18:05 xenpub1dc1 xendomains: Error: Internal Server Error
> Jan 28 22:18:05 xenpub1dc1 xendomains:  failed
> Jan 28 22:18:05 xenpub1dc1 xendomains:
> Jan 28 22:18:05 xenpub1dc1 rc: Stopping xendomains:  failed
> Jan 28 22:18:05 xenpub1dc1 kernel: Kernel logging (proc) stopped.
> Jan 28 22:18:05 xenpub1dc1 kernel: Kernel log daemon terminating.
> Jan 28 22:18:06 xenpub1dc1 syslog: klogd shutdown succeeded
> Jan 28 22:18:06 xenpub1dc1 exiting on signal 15
> Jan 28 22:20:06 xenpub1dc1 syslogd 1.4.1: restart.
> Jan 28 22:20:06 xenpub1dc1 syslog: syslogd startup succeeded
> Jan 28 22:20:06 xenpub1dc1 kernel: klogd 1.4.1, log source = /proc/kmsg
> started.
> Jan 28 22:20:06 xenpub1dc1 kernel: Linux version 2.6.11.12-xen0
> (root@xenpub1dc1.atlasgroup.net) (gcc version 3.4.4 20050721 (Red Hat
> 3.4.4-2)) #4 Thu Jan
> File messages not changed so no update needed.
> [root@xenpub1dc1 log]# ls
> acpid            boot.log.1  cron.2   mail       messages    prelink.log
> samba     spooler    wtmp            xfrd.log
> anaconda.log     boot.log.2  cups     maillog    messages.1  rpmpkgs
> secure    spooler.1  xend-debug.log  Xorg.0.log
> anaconda.syslog  cron        dmesg    maillog.1  messages.2  rpmpkgs.1
> secure.1  spooler.2  xend.log        Xorg.0.log.old
> boot.log         cron.1      lastlog  maillog.2  ppp         rpmpkgs.2
> secure.2  vbox       xend.log.1      yum.log
> [root@xenpub1dc1 log]# joe xend-debug.log
> Processing ''/etc/joe/joerc''...done
> Processing ''/etc/joe/joerc''...done
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>     I    xend-debug.log
> Row 171  Col 1   11:02  Ctrl-K H for help
> recv_fe_driver_status>
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 185025,
''rx_shmem_frame'':
> 185026, ''handle'': 0}
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 185029,
''rx_shmem_frame'':
> 185030, ''handle'': 1}
> {}
> {}
> VirqClient.virqReceived> 4
> vif-bridge down vif=vif8.0 domain=niagras1 mac=02:00:00:01:02:00
> bridge=xen-br0
> vif-bridge down vif=vif8.1 domain=niagras1 mac=aa:00:00:53:7a:5b
> bridge=xen-br0
> VirqClient.virqReceived> 4
> vif-bridge down vif=vif7.0 domain=vpn1dc1 mac=02:00:00:01:01:00
> bridge=xen-br0
> vif-bridge down vif=vif7.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> -bridge down vif=vif4.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> interface vif4.1 does not exist!
> VIRTUAL MEMORY ARRANGEMENT:
>  Loaded kernel: c0100000->c02c9df4
>  Init. ramdisk: c02ca000->c02ca000
>  Phys-Mach map: c02ca000->c02e2000
>  Page tables:   c02e2000->c02e4000
>  Start info:    c02e4000->c02e5000
>  Boot stack:    c02e5000->c02e6000
>  TOTAL:         c0000000->c0400000
>  ENTRY ADDRESS: c0100000
> vif-bridge up vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
> bridge=xen-br0
> interface vif5.0 does not exist!
> vif-bridge up vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> interface vif5.1 does not exist!
> recv_fe_driver_status> {''status'': 1}
> 
> recv_fe_driver_status>
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 202459,
''rx_shmem_frame'':
> 202460, ''handle'': 0}
> 
> recv_fe_interface_connect {''tx_shmem_frame'': 202463,
''rx_shmem_frame'':
> 202464, ''handle'': 1}
> VirqClient.virqReceived> 4
> vif-bridge down vif=vif5.0 domain=vpn1dc1 mac=02:00:00:01:01:00
> bridge=xen-br0
> interface vif5.0 does not exist!
> vif-bridge down vif=vif5.1 domain=vpn1dc1 mac=02:00:00:01:01:01
> bridge=xen-br1
> interface vif5.1 does not exist!
> 
> File xend-debug.log not changed so no update needed.
> [root@xenpub1dc1 log]# ls
> acpid            boot.log.1  cron.2   mail       messages    prelink.log
> samba     spooler    wtmp            xfrd.log
> anaconda.log     boot.log.2  cups     maillog    messages.1  rpmpkgs
> secure    spooler.1  xend-debug.log  Xorg.0.log
> anaconda.syslog  cron        dmesg    maillog.1  messages.2  rpmpkgs.1
> secure.1  spooler.2  xend.log        Xorg.0.log.old
> boot.log         cron.1      lastlog  maillog.2  ppp         rpmpkgs.2
> secure.2  vbox       xend.log.1      yum.log
> [root@xenpub1dc1 log]# joe xend.log
> Processing ''/etc/joe/joerc''...done
> Processing ''/etc/joe/joerc''...done
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>     I    xend.log
> Row 6157 Col 100 11:07  Ctrl-K H for help
> [2006-01-28 22:06:36 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:36 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:36 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:36 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:36 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:36 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:36 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:38 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:38 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:38 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:38 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:38 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:38 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:38 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:40 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:40 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:40 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:40 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:40 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:40 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:40 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:42 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:42 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:42 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:42 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:42 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:42 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:42 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:44 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:44 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:44 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:44 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:44 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:44 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:44 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:46 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:46 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:46 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:46 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:46 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:46 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:46 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:48 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:48 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:48 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:48 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:48 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:48 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:48 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:50 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:50 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:50 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:50 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:50 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:50 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:50 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:52 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:52 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:52 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:52 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:52 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:52 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:52 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:54 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:54 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:54 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:54 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:54 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:54 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:54 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:56 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:56 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:56 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:56 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1
> [2006-01-28 22:06:56 xend] DEBUG (XendDomainInfo:634) Closing console,
> domain 2
> [2006-01-28 22:06:56 xend] INFO (XendRoot:113) EVENT> xend.domain.exit
> [''vpn1dc1'', ''2'',
''poweroff'']
> [2006-01-28 22:06:56 xend] INFO (XendRoot:113) EVENT>
> xend.domain.destroy [''vpn1dc1'', ''2'']
> [2006-01-28 22:06:57 xend] DEBUG (XendDomain:244) XendDomain>reap>
> domain died name=vpn1dc1 id=2
> [2006-01-28 22:06:57 xend] DEBUG (XendDomain:247) XendDomain>reap>
> shutdown id=2 reason=poweroff
> [2006-01-28 22:06:57 xend] DEBUG (XendDomain:487)
> domain_restart_schedule> 2 poweroff 0
> [2006-01-28 22:06:57 xend] INFO (XendDomain:568) Destroying domain:
> name=vpn1dc1 
> etc., etc., etc.
> 
> Here are the configuration files for the two domUs:
> #  -*- mode: python; -*-
>
#===========================================================================>
# Python configuration setup for ''xm create''.
> # This script sets the parameters used when a domain is created using
> ''xm create''.
> # You use a separate script for each domain you want to create, or 
> # you can set the parameters for the domain on the xm command line.
>
#===========================================================================>
>
#----------------------------------------------------------------------------
> # Kernel image file.
> kernel = "/boot/vmlinuz-2.4.30-xenUVPN"
> 
> # Optional ramdisk.
> #ramdisk = "/boot/initrd.gz"
> 
> # The domain build function. Default is ''linux''.
> #builder=''linux''
> 
> # Initial memory allocation (in megabytes) for the new domain.
> memory = 96
> 
> # A name for your domain. All domains must have different names.
> name = "niagras1"
> 
> # Which CPU to start domain on? 
> #cpu = -1   # leave to Xen to pick
> 
>
#----------------------------------------------------------------------------
> # Define network interfaces.
> 
> # Number of network interfaces. Default is 1.
> nics=2
> 
> # Optionally define mac and/or bridge for the network interfaces.
> # Random MACs are assigned if not given.
> #vif = [ ''mac=aa:00:00:00:00:11, bridge=xen-br0'' ]
> 
> vif = [''mac=02:00:00:01:02:00, bridge=xen-br0'' ]
> 
>
#----------------------------------------------------------------------------
> # Define the disk devices you want the domain to have access to, and
> # what you want them accessible as.
> # Each disk entry is of the form phy:UNAME,DEV,MODE
> # where UNAME is the device, DEV is the device name the domain will see,
> # and MODE is r for read-only, w for read-write.
> 
> #disk = [ ''phy:hda1,hda1,w'' ]
> 
> disk = [ ''phy:VG1/niagras1root,sda1,w'',
''phy:VG1/niagras1swap,sda2,w'' ]
> 
>
#----------------------------------------------------------------------------
> # Set the kernel command line for the new domain.
> # You only need to define the IP parameters and hostname if the
domain''s
> # IP config doesn''t, e.g. in ifcfg-eth0 or via DHCP.
> # You can use ''extra'' to set the runlevel and custom
environment
> # variables used by custom rc scripts (e.g. VMID=, usr= ).
> 
> # Set if you want dhcp to allocate the IP address.
> #dhcp="dhcp"
> # Set netmask.
> #netmask> # Set default gateway.
> #gateway> # Set the hostname.
> #hostname= "vm%d" % vmid
> 
> # Set root device.
> root = "/dev/sda1 ro"
> 
> # Root device for nfs.
> #root = "/dev/nfs"
> # The nfs server.
> #nfs_server = ''169.254.1.0''  
> # Root directory on the nfs server.
> #nfs_root   = ''/full/path/to/root/directory''
> 
> # Sets runlevel 4.
> #extra = "4"
> 
>
#----------------------------------------------------------------------------
> # Set according to whether you want the domain restarted when it exits.
> # The default is ''onreboot'', which restarts the domain
when it shuts
> down
> # with exit code reboot.
> # Other values are ''always'', and
''never''.
> 
> #restart = ''onreboot''
> 
>
#===========================================================================>
> 
> #  -*- mode: python; -*-
>
#===========================================================================>
# Python configuration setup for ''xm create''.
> # This script sets the parameters used when a domain is created using
> ''xm create''.
> # You use a separate script for each domain you want to create, or 
> # you can set the parameters for the domain on the xm command line.
>
#===========================================================================>
>
#----------------------------------------------------------------------------
> # Kernel image file.
> kernel = "/boot/vmlinuz-2.4.30-xenUVPN"
> 
> # Optional ramdisk.
> #ramdisk = "/boot/initrd.gz"
> 
> # The domain build function. Default is ''linux''.
> #builder=''linux''
> 
> # Initial memory allocation (in megabytes) for the new domain.
> memory = 96
> 
> # A name for your domain. All domains must have different names.
> name = "vpn1dc1"
> 
> # Which CPU to start domain on? 
> #cpu = -1   # leave to Xen to pick
> 
>
#----------------------------------------------------------------------------
> # Define network interfaces.
> 
> # Number of network interfaces. Default is 1.
> nics=2
> 
> # Optionally define mac and/or bridge for the network interfaces.
> # Random MACs are assigned if not given.
> #vif = [ ''mac=aa:00:00:00:00:11, bridge=xen-br0'' ]
> 
> vif = [''mac=02:00:00:01:01:00, bridge=xen-br0'',
''mac=02:00:00:01:01:01,
> bridge=xen-br1'' ]
> 
>
#----------------------------------------------------------------------------
> # Define the disk devices you want the domain to have access to, and
> # what you want them accessible as.
> # Each disk entry is of the form phy:UNAME,DEV,MODE
> # where UNAME is the device, DEV is the device name the domain will see,
> # and MODE is r for read-only, w for read-write.
> 
> #disk = [ ''phy:hda1,hda1,w'' ]
> 
> disk = [ ''phy:VG1/vpn1dc1root,sda1,w'',
''phy:VG1/vpn1dc1swap,sda2,w'' ]
> 
>
#----------------------------------------------------------------------------
> # Set the kernel command line for the new domain.
> # You only need to define the IP parameters and hostname if the
domain''s
> # IP config doesn''t, e.g. in ifcfg-eth0 or via DHCP.
> # You can use ''extra'' to set the runlevel and custom
environment
> # variables used by custom rc scripts (e.g. VMID=, usr= ).
> 
> # Set if you want dhcp to allocate the IP address.
> #dhcp="dhcp"
> # Set netmask.
> #netmask> # Set default gateway.
> #gateway> # Set the hostname.
> #hostname= "vm%d" % vmid
> 
> # Set root device.
> root = "/dev/sda1 ro"
> 
> # Root device for nfs.
> #root = "/dev/nfs"
> # The nfs server.
> #nfs_server = ''169.254.1.0''  
> # Root directory on the nfs server.
> #nfs_root   = ''/full/path/to/root/directory''
> 
> # Sets runlevel 4.
> #extra = "4"
> 
>
#----------------------------------------------------------------------------
> # Set according to whether you want the domain restarted when it exits.
> # The default is ''onreboot'', which restarts the domain
when it shuts
> down
> # with exit code reboot.
> # Other values are ''always'', and
''never''.
> 
> #restart = ''onreboot''
> 
>
#===========================================================================>
> 
> I''m stumped but it sure looks serious.  Have I produced some
dreadful
> misconfiguration? This host is being shipped to a remote data center on
> Monday and goes into production in a week.  Any help would be greatly
> appreciated.  Thanks - John
We are also using a very slightly customized network script since we
have three interfaces.  Here is the network script:

# Exit if anything goes wrong.
set -e 

# First arg is the operation.
OP=$1
shift

# Pull variables in args in to environment.
for arg ; do export "${arg}" ; done

bridge=${bridge:-xen-br0}
netdev=${netdev:-eth0}
antispoof=${antispoof:-yes}
SCRIPTDIR=/etc/xen/scripts

echo "network $OP bridge=$bridge netdev=$netdev antispoof=$antispoof"

# Usage: transfer_addrs src dst
# Copy all IP addresses (including aliases) from device $src to device
$dst.
transfer_addrs () {
    local src=$1
    local dst=$2
    # Don''t bother if $dst already has IP addresses.
    if ip addr show dev ${dst} | egrep -q ''^ *inet '' ; then
        return
    fi
    # Address lines start with ''inet'' and have the device in
them.
    # Replace ''inet'' with ''ip addr add'' and
change the device name $src
    # to ''dev $src''. Remove netmask as we''ll add
routes later.
    ip addr show dev ${src} | egrep ''^ *inet '' | sed -e "
s/inet/ip addr add/
s@\([0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+\)/[0-9]\+@\1@
s/${src}/dev ${dst}/
" | sh -e
}

# Usage: del_addrs src
del_addrs () {
    local src=$1
    ip addr show dev ${src} | egrep ''^ *inet '' | sed -e "
s/inet/ip addr del/
s@\([0-9]\+\.[0-9]\+\.[0-9]\+\.[0-9]\+\)/[0-9]\+@\1@
s/${src}/dev ${src}/
" | sh -e
}

# Usage: transfer_routes src dst
# Get all IP routes to device $src, delete them, and
# add the same routes to device $dst.
# The original routes have to be deleted, otherwise adding them
# for $dst fails (duplicate routes).
transfer_routes () {
    local src=$1
    local dst=$2
    # List all routes and grep the ones with $src in.
    # Stick ''ip route del'' on the front to delete.
    # Change $src to $dst and use ''ip route add'' to add.
    ip route list | grep ${src} | sed -e "
h
s/^/ip route del /
P
g
s/${src}/${dst}/
s/^/ip route add /
P
d
" | sh -e
}

# Usage: create_bridge bridge
create_bridge () {
    local bridge=$1

    # Don''t create the bridge if it already exists.
    if ! brctl show | grep -q ${bridge} ; then
        brctl addbr ${bridge}
        brctl stp ${bridge} off
        brctl setfd ${bridge} 0
    fi
    ifconfig ${bridge} up
}

# Usage: add_to_bridge bridge dev
add_to_bridge () {
    local bridge=$1
    local dev=$2
    # Don''t add $dev to $bridge if it''s already on a bridge.
    if ! brctl show | grep -q ${dev} ; then
        brctl addif ${bridge} ${dev}
    fi
}

# Usage: antispoofing dev bridge
# Set the default forwarding policy for $dev to drop.
# Allow forwarding to the bridge.
antispoofing () {
    local dev=$1
    local bridge=$2

    iptables -P FORWARD DROP
    iptables -A FORWARD -m physdev --physdev-in ${dev} -j ACCEPT
}

# Usage: show_status dev bridge
# Print ifconfig and routes.
show_status () {
    local dev=$1
    local bridge=$2
    
    echo
''============================================================''
    ifconfig ${dev}
    ifconfig ${bridge}
    echo '' ''
    ip route list
    echo '' ''
    route -n
    echo
''============================================================''
}

op_start () {
    if [ "${bridge}" == "null" ] ; then
        return
    fi
# Create the bridge and give it the interface IP addresses.
# Move the interface routes onto the bridge.
    while read bridge netdev
	do
            create_bridge ${bridge}
            if [ -n "$(ip add ls dev $netdev | grep ''inet
'')" ]; then
                transfer_addrs ${netdev} ${bridge}
                transfer_routes ${netdev} ${bridge}
            fi
# Don''t add $dev to $bridge if it''s already on a bridge.
            if ! brctl show | grep -q ${netdev} ; then
                brctl addif ${bridge} ${netdev}
            fi
            
            if [ ${antispoof} == ''yes'' ] ; then
                antispoofing ${netdev} ${bridge}
            fi
    done < ${SCRIPTDIR}/bridges.txt
}

op_stop () {
    if [ "${bridge}" == "null" ] ; then
        return
    fi
# Remove the interface from the bridge.
# Move the routes back to the interface.
    while read bridge netdev
    do
        brctl delif ${bridge} ${netdev}              
        if [ -n "$(ip add ls dev $netdev | grep ''inet
'')" ]; then
            transfer_routes ${bridge} ${netdev}
        fi
    done < ${SCRIPTDIR}/bridges.txt
# It''s not our place to be enabling forwarding...
}

case ${OP} in
  start)
        op_start
        ;;
    
    stop)
        op_stop
        ;;

    status)
        show_status ${netdev} ${bridge}
       ;;

    *)
       echo ''Unknown command: '' ${OP}
       echo ''Valid commands are: start, stop, status''
       exit 1
esac

Here is the bridges.txt file:
xen-br0 eth0
xen-br1 eth1
xen-br2 eth2

-- 
John A. Sullivan III
Open Source Development Corporation
+1 207-985-7880
jsullivan@opensourcedevel.com

Financially sustainable open source development
http://www.opensourcedevel.com


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

John,

Did you ever find a solution to this problem?  I am experiencing the exact 
same problem (xendomains stop hangs almost always during shutdown, but 
succeeds if called from a terminal before shutdown).

Athanasios

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

John,

Did you ever find a solution to this problem?  I am experiencing the exact 
same problem (xendomains stop hangs almost always during shutdown, but 
succeeds if called from a terminal before shutdown).

Athanasios

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Tue, 2006-02-21 at 14:40 +0200, Athanasios Floros
wrote:
> John,
> 
> Did you ever find a solution to this problem?  I am experiencing the exact 
> same problem (xendomains stop hangs almost always during shutdown, but 
> succeeds if called from a terminal before shutdown).
> 
> Athanasios
<snip>
No, I''ve received neither an answer nor a resolution and I am quite
concerned since we are running this in production.

Is anyone else experiencing this? - John
-- 
John A. Sullivan III
Open Source Development Corporation
+1 207-985-7880
jsullivan@opensourcedevel.com

If you would like to participate in the development of an open source
enterprise class network security management system, please visit
http://iscs.sourceforge.net


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Dear John,

I think I may have good news for you.  I have not received an answer or a 
solution either, however I made some modifications in my system that work out 
fine so far.  However, I have not rebooted my system too many times since the 
modifications, so I can''t be sure that it will be stable in the long
term
-but the system does shutdown/reboot flawlessly since the modifications.

I was experiencing the problem with the default Xen installation of Suse 
Professional 9.3 (including the updates available through YOU).  This means 
that I was using Xen version 2.0.5c-4 with kernel 2.6.11.4-21.10.

I removed these packages form my system and installed the latest version of 
Xen (and kernel) from the packages found here:

http://www.suse.de/~garloff/linux/xen/

So in effect, I now use Xen 2.0.7 and kernel 2.6.11.4.21-8 (both in dom0 and 
in the unprivileged domains).  This seems to have solved the problems during 
shutdown/reboot, despite the fact that the kernel is somewhat older version 
that the one officially suplied by Suse.

The only unfortunate side effect of this setup, is that YOU now reports that 
"there are security updates available", because there is a newer Xen
kernel
available for my system.  I will put up with this and will not try a kernel 
update, as long as my current setup works without any problems.

I understand that you would not like to experiment a lot with your production 
system, so if I find out about any issues with this solution I will do my 
best to keep you posted.

Kind regards,
Athanasios


On Saturday 25 February 2006 08:44, John A. Sullivan III
wrote:
> On Tue, 2006-02-21 at 14:40 +0200, Athanasios Floros wrote:
> > John,
> >
> > Did you ever find a solution to this problem?  I am experiencing the
> > exact same problem (xendomains stop hangs almost always during
shutdown,
> > but succeeds if called from a terminal before shutdown).
> >
> > Athanasios
>
> <snip>
> No, I''ve received neither an answer nor a resolution and I am
quite
> concerned since we are running this in production.
>
> Is anyone else experiencing this? - John
-- 
In case I don''t see you:
Good afternoon, good evening and good night.-

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Thanks, Athanasios.  I am running xen-2.0.7 with 2.6.11.12 in dom0 and
2.4.31, I believe, in the problematic domUs.  Interesting that it might
be a kernel problem.  Alas, mine are heavily patched so I hesitate to
change them - John

On Sat, 2006-02-25 at 12:20 +0200, Athanasios Floros
wrote:
> Dear John,
> 
> I think I may have good news for you.  I have not received an answer or a 
> solution either, however I made some modifications in my system that work
out
> fine so far.  However, I have not rebooted my system too many times since
the
> modifications, so I can''t be sure that it will be stable in the
long term
> -but the system does shutdown/reboot flawlessly since the modifications.
> 
> I was experiencing the problem with the default Xen installation of Suse 
> Professional 9.3 (including the updates available through YOU).  This means
> that I was using Xen version 2.0.5c-4 with kernel 2.6.11.4-21.10.
> 
> I removed these packages form my system and installed the latest version of
> Xen (and kernel) from the packages found here:
> 
> http://www.suse.de/~garloff/linux/xen/
> 
> So in effect, I now use Xen 2.0.7 and kernel 2.6.11.4.21-8 (both in dom0
and
> in the unprivileged domains).  This seems to have solved the problems
during
> shutdown/reboot, despite the fact that the kernel is somewhat older version
> that the one officially suplied by Suse.
> 
> The only unfortunate side effect of this setup, is that YOU now reports
that
> "there are security updates available", because there is a newer
Xen kernel
> available for my system.  I will put up with this and will not try a kernel
> update, as long as my current setup works without any problems.
> 
> I understand that you would not like to experiment a lot with your
production
> system, so if I find out about any issues with this solution I will do my 
> best to keep you posted.
> 
> Kind regards,
> Athanasios
> 
> 
> On Saturday 25 February 2006 08:44, John A. Sullivan III wrote:
> > On Tue, 2006-02-21 at 14:40 +0200, Athanasios Floros wrote:
> > > John,
> > >
> > > Did you ever find a solution to this problem?  I am experiencing
the
> > > exact same problem (xendomains stop hangs almost always during
shutdown,
> > > but succeeds if called from a terminal before shutdown).
> > >
> > > Athanasios
> >
> > <snip>
> > No, I''ve received neither an answer nor a resolution and I am
quite
> > concerned since we are running this in production.
> >
> > Is anyone else experiencing this? - John
> 
-- 
John A. Sullivan III
Open Source Development Corporation
+1 207-985-7880
jsullivan@opensourcedevel.com

Financially sustainable open source development
http://www.opensourcedevel.com


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

How long did you wait for "xendomains stop" to shut down your domUs? 
This
morning I thought that it hung again during shutdown, but I deliberately 
waited for an unusual amount of time (around 5 minutes I believe, maybe more) 
and it did complete successfully.  This was the first time I tried waiting so 
long, while there was no disk activity or other clues that the system was not 
in an endless loop.

Too bad I never tried this with the default Suse installation...

Athanasios

On Saturday 25 February 2006 18:49, you wrote:
> Thanks, Athanasios.  I am running xen-2.0.7 with 2.6.11.12 in dom0 and
> 2.4.31, I believe, in the problematic domUs.  Interesting that it might
> be a kernel problem.  Alas, mine are heavily patched so I hesitate to
> change them - John
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

It usually does eventually just as you found.  However, that long wait
appears to be something seriously wrong in shutting down the domUs.  I
don''t know if the python script eventually dies and then the shutdown
can proceed or what is happening.  I haven''t taken the time to dig that
deep - John

On Sat, 2006-02-25 at 19:41 +0200, Athanasios Floros
wrote:
> How long did you wait for "xendomains stop" to shut down your
domUs?  This
> morning I thought that it hung again during shutdown, but I deliberately 
> waited for an unusual amount of time (around 5 minutes I believe, maybe
more)
> and it did complete successfully.  This was the first time I tried waiting
so
> long, while there was no disk activity or other clues that the system was
not
> in an endless loop.
> 
> Too bad I never tried this with the default Suse installation...
> 
> Athanasios
> 
> On Saturday 25 February 2006 18:49, you wrote:
> > Thanks, Athanasios.  I am running xen-2.0.7 with 2.6.11.12 in dom0 and
> > 2.4.31, I believe, in the problematic domUs.  Interesting that it
might
> > be a kernel problem.  Alas, mine are heavily patched so I hesitate to
> > change them - John
-- 
John A. Sullivan III
Open Source Development Corporation
+1 207-985-7880
jsullivan@opensourcedevel.com

Financially sustainable open source development
http://www.opensourcedevel.com


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users