Hello,
I am running xen-3.3.1 with rhel5.3 dom0.
I have a rhel5.3 domU that is stuck in the boot process. It seems it is
trying to bring up the network. We see this problem only occassionally.
This is the last console output:
Running configure_os...
Warning: Configuring networking...
Warning: HOSTNAME is not defined
Updating ifcfg-eth0 [ OK ]
Allowing IPv4 link local addresses [ OK ]
Warning: Starting networking...
Bringing up loopback interface: [ OK ]
It has been stuck for a few days now. I did a dump-core on it and it seems
to be doing network start -> ifup-eth ifcfg-eth0 boot -> ip link set dev
eth0 up -> hypercall. I''m somewhat assuming it made the hypercall
but never
got the event back but I''m not sure as I''m new to this. Does
anyone have a
good pointers on how exactly hypercalls work or any info on looking at
event-channels? Any ideas why it might be stuck?
PID: 1290 TASK: ffff88007f7b50c0 CPU: 0 COMMAND: "ip"
ARG: ip link set dev eth0 up
ENV: TERM=linux
OLDPWD=/etc/sysconfig/network-scripts
PATH=/sbin:/usr/sbin:/bin:/usr/bin
PWD=/etc/sysconfig/network-scripts
LANG=en_US.UTF-8
SHLVL=2
_=/sbin/ip
Backtrace:
PID: 1290 TASK: ffff88007f7b50c0 CPU: 0 COMMAND: "ip"
#0 [ffff88007c135d08] schedule at ffffffff80261e17
#1 [ffff88007c135d50] hypercall_page at ffffffff8020622a
#2 [ffff88007c135d78] hypercall_page at ffffffff8020622a
#3 [ffff88007c135da0] __netif_rx_schedule at ffffffff80410360
#4 [ffff88007c135dc0] network_open at ffffffff8810c992
#5 [ffff88007c135de0] dev_open at ffffffff80411cd2
#6 [ffff88007c135df0] dev_change_flags at ffffffff80410263
#7 [ffff88007c135e20] devinet_ioctl at ffffffff80443763
#8 [ffff88007c135ec0] sock_ioctl at ffffffff80408968
#9 [ffff88007c135ee0] do_ioctl at ffffffff802437ba
#10 [ffff88007c135f00] vfs_ioctl at ffffffff80231010
#11 [ffff88007c135f40] sys_ioctl at ffffffff8024ddde
#12 [ffff88007c135f80] system_call at ffffffff8025f106
RIP: 00002b5bbbf43c77 RSP: 00007fffef0627f8 RFLAGS: 00000246
RAX: 0000000000000010 RBX: ffffffff8025f106 RCX: ffffffff8025f080
RDX: 00007fffef062830 RSI: 0000000000008914 RDI: 0000000000000006
RBP: 00007fffef062830 R8: 00007fffef062834 R9: 0000000000000003
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000001
R13: 0000000000000001 R14: 0000000000000006 R15: 0000000000000000
ORIG_RAX: 0000000000000010 CS: e033 SS: e02b
>From dom0 it looks like the network is setup right for the vif, they are
definitely all dropping packets right now though but I figured that was due
to the domU not finishing its setup:
vif912.0 Link encap:Ethernet HWaddr FE:FF:FF:FF:FF:FF
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:109 errors:0 dropped:785242 overruns:0 carrier:0
collisions:0 txqueuelen:32
RX bytes:0 (0.0 b) TX bytes:7501 (7.3 KiB)
Thanks,
Tom
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
On Thu, Jun 17, 2010 at 4:31 AM, Thomas Graves <tgraves@yahoo-inc.com> wrote:> Hello, > > I am running xen-3.3.1 with rhel5.3 dom0. > I have a rhel5.3 domU that is stuck in the boot process. It seems it is > trying to bring up the network. We see this problem only occassionally.So it''s not login console not displayed, only something that happens occasionally?> It has been stuck for a few days now. I did a dump-core on it and it seems > to be doing network start -> ifup-eth ifcfg-eth0 boot -> ip link set dev > eth0 up -> hypercall. I''m somewhat assuming it made the hypercall but never > got the event back but I''m not sure as I''m new to this. Does anyone have a > good pointers on how exactly hypercalls work or any info on looking at > event-channels? Any ideas why it might be stuck?I never had this problem, but here''s several things you might want to try: - udate your OS. RHEL 5.5. is out already. At least update your kernel-xen rpm. - update your Xen version. Xen 3.4.2 (the easy way is from Gitco) has been quite stable for me. If you want to try latest version, you could also test Xen 4.0 -- Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On 6/16/10 9:58 PM, "Fajar A. Nugraha" <fajar@fajar.net> wrote: On Thu, Jun 17, 2010 at 4:31 AM, Thomas Graves <tgraves@yahoo-inc.com> wrote:> Hello, > > I am running xen-3.3.1 with rhel5.3 dom0. > I have a rhel5.3 domU that is stuck in the boot process. It seems it is > trying to bring up the network. We see this problem only occassionally.So it''s not login console not displayed, only something that happens occasionally? Right, I''ve seen it happen 3 times in the past month or so. It will start to boot and I can see the console up to the the line ( Bringing up loopback interface: [ OK ]) and then the console stops and it never gets to the login prompt and the network isn''t up yet so I can''t get into it.> It has been stuck for a few days now. I did a dump-core on it and it seems > to be doing network start -> ifup-eth ifcfg-eth0 boot -> ip link set dev > eth0 up -> hypercall. I''m somewhat assuming it made the hypercall but never > got the event back but I''m not sure as I''m new to this. Does anyone have a > good pointers on how exactly hypercalls work or any info on looking at > event-channels? Any ideas why it might be stuck?I never had this problem, but here''s several things you might want to try: - udate your OS. RHEL 5.5. is out already. At least update your kernel-xen rpm. - update your Xen version. Xen 3.4.2 (the easy way is from Gitco) has been quite stable for me. If you want to try latest version, you could also test Xen 4.0 I actually just upgraded a box to rhel5.4 and xen3.4.2 so I''ll try it there to see if I see it but since it doesn''t happen very often probably won''t know for sure for a while. Any pointers on good reads on debugging the events or ring buffers? Like how to map xenstore info to something useful to look at: tx-ring-ref = "768" rx-ring-ref = "769" event-channel = "14" Thanks again Tom _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Sorry my reply formatting got messed up, reposting with it fixed. On Thu, Jun 17, 2010 at 4:31 AM, Thomas Graves <tgraves@yahoo-inc.com> wrote:>>Hello, >> >>I am running xen-3.3.1 with rhel5.3 dom0. >>I have a rhel5.3 domU that is stuck in the boot process. It seems it is >> trying to bring up the network. We see this problem only occassionally.> So it''s not login console not displayed, only something that happens > occasionally?Right, I''ve seen it happen 3 times in the past month or so. It will start to boot and I can see the console up to the the line ( Bringing up loopback interface: [ OK ]) and then the console stops and it never gets to the login prompt and the network isn''t up yet so I can''t get into it.>> It has been stuck for a few days now. I did a dump-core on it and it seems >> to be doing network start -> ifup-eth ifcfg-eth0 boot -> ip link set dev >> eth0 up -> hypercall. I''m somewhat assuming it made the hypercall but never >> got the event back but I''m not sure as I''m new to this. Does anyone have a >> good pointers on how exactly hypercalls work or any info on looking at >> event-channels? Any ideas why it might be stuck?> I never had this problem, but here''s several things you might want to try: > - udate your OS. RHEL 5.5. is out already. At least update your kernel-xen rpm. > - update your Xen version. Xen 3.4.2 (the easy way is from Gitco) has > been quite stable for me. If you want to try latest version, you could > also test Xen 4.0I actually just upgraded a box to rhel5.4 and xen3.4.2 so I''ll try it there to see if I see it but since it doesn''t happen very often probably won''t know for sure for a while. Any pointers on good reads on debugging the events or ring buffers? Like how to map xenstore info to something useful to look at: tx-ring-ref = "768" rx-ring-ref = "769" event-channel = "14" Thanks again Tom _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I upgraded to xen-3.4.3 and rhel5.4 and it actually seems to happen more often. It does hang at a different point though. All the console shows is "console tty=/dev/pts/1". No one has seen this? Tom On 6/17/10 9:09 AM, "Thomas Graves" <tgraves@yahoo-inc.com> wrote: Sorry my reply formatting got messed up, reposting with it fixed. On Thu, Jun 17, 2010 at 4:31 AM, Thomas Graves <tgraves@yahoo-inc.com> wrote:>>Hello, >> >>I am running xen-3.3.1 with rhel5.3 dom0. >>I have a rhel5.3 domU that is stuck in the boot process. It seems it is >> trying to bring up the network. We see this problem only occassionally.> So it''s not login console not displayed, only something that happens > occasionally?Right, I''ve seen it happen 3 times in the past month or so. It will start to boot and I can see the console up to the the line ( Bringing up loopback interface: [ OK ]) and then the console stops and it never gets to the login prompt and the network isn''t up yet so I can''t get into it.>> It has been stuck for a few days now. I did a dump-core on it and it seems >> to be doing network start -> ifup-eth ifcfg-eth0 boot -> ip link set dev >> eth0 up -> hypercall. I''m somewhat assuming it made the hypercall but never >> got the event back but I''m not sure as I''m new to this. Does anyone have a >> good pointers on how exactly hypercalls work or any info on looking at >> event-channels? Any ideas why it might be stuck?> I never had this problem, but here''s several things you might want to try: > - udate your OS. RHEL 5.5. is out already. At least update your kernel-xen rpm. > - update your Xen version. Xen 3.4.2 (the easy way is from Gitco) has > been quite stable for me. If you want to try latest version, you could > also test Xen 4.0I actually just upgraded a box to rhel5.4 and xen3.4.2 so I''ll try it there to see if I see it but since it doesn''t happen very often probably won''t know for sure for a while. Any pointers on good reads on debugging the events or ring buffers? Like how to map xenstore info to something useful to look at: tx-ring-ref = "768" rx-ring-ref = "769" event-channel = "14" Thanks again Tom _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users