Jody Belka
2009-Aug-04 14:02 UTC
[Xen-users] Xen Panic w/ dom0 2.6.30.2 (gentoo-xen-kernel.googlecode.com patches)
Hi, I just tried upgrading my dom0 to 2.6.30.2, using the patches on gentoo-xen-kernel.googlecode.com. It built ok, but trying to boot results in a "FATAL PAGE FAULT" error from xen. Xen specs = Xen 3.4.1-rc2, x84_64 Has anyone seen this before, and maybe has an idea what''s going wrong, and what I need to do to get a working dom0? I''ve attached screengrabs of the panic stack/call trace. Jody _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Christian Fischer
2009-Aug-05 13:34 UTC
Re: [Xen-users] Xen Panic w/ dom0 2.6.30.2 (gentoo-xen-kernel.googlecode.com patches)
On Tuesday 04 August 2009, Jody Belka wrote:> Hi, > > I just tried upgrading my dom0 to 2.6.30.2, using the patches on > gentoo-xen-kernel.googlecode.com. It built ok, but trying to boot > results in a "FATAL PAGE FAULT" error from xen. > > Xen specs = Xen 3.4.1-rc2, x84_64 > > Has anyone seen this before, and maybe has an idea what''s going wrong, > and what I need to do to get a working dom0? I''ve attached screengrabs > of the panic stack/call trace. > > > JodyTried 2.6.30-r2 on HP ProLiant DL380G5 with xen-3.3, panics, no output on vga after relinquishing VGA console, works on Dell Optiplex Core2Duo. Switched back to 2.6.29-r4, works out of the box. Christian -- "Without music to decorate it, time is just a bunch of boring production deadlines or dates by which bills must be paid." --- Frank Vincent Zappa _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Jody Belka
2009-Aug-05 22:51 UTC
Re: [Xen-users] Xen Panic w/ dom0 2.6.30.2 (gentoo-xen-kernel.googlecode.com patches)
On Wed, Aug 05, 2009 at 03:34:17PM +0200, Christian Fischer wrote:> On Tuesday 04 August 2009, Jody Belka wrote: > > Hi, > > > > I just tried upgrading my dom0 to 2.6.30.2, using the patches on > > gentoo-xen-kernel.googlecode.com. It built ok, but trying to boot > > results in a "FATAL PAGE FAULT" error from xen. > > > > Xen specs = Xen 3.4.1-rc2, x84_64 > > > > Has anyone seen this before, and maybe has an idea what''s going wrong, > > and what I need to do to get a working dom0? I''ve attached screengrabs > > of the panic stack/call trace. > > > > > > Jody > > Tried 2.6.30-r2 on HP ProLiant DL380G5 with xen-3.3, panics, no output on vga > after relinquishing VGA console, works on Dell Optiplex Core2Duo. > > Switched back to 2.6.29-r4, works out of the box.Ok, built 2.6.29.6, with the v6 gentoo-xen-kernel patches, and boots fine. So I can actually use this system again now :) Tried upgrading in the first place because of a bug in plain 2.6.29 that just recently started catching me out, causing endless BUGs from linux. Since -stable on .29 fixes it as well though, this solution is workable. I guess the .30 patches just aren''t quite there yet, oh well. J _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Andrew Lyon
2009-Aug-06 07:40 UTC
Re: [Xen-users] Xen Panic w/ dom0 2.6.30.2 (gentoo-xen-kernel.googlecode.com patches)
On Wed, Aug 5, 2009 at 11:51 PM, Jody Belka<lists-xen@pimb.org> wrote:> On Wed, Aug 05, 2009 at 03:34:17PM +0200, Christian Fischer wrote: >> On Tuesday 04 August 2009, Jody Belka wrote: >> > Hi, >> > >> > I just tried upgrading my dom0 to 2.6.30.2, using the patches on >> > gentoo-xen-kernel.googlecode.com. It built ok, but trying to boot >> > results in a "FATAL PAGE FAULT" error from xen. >> > >> > Xen specs = Xen 3.4.1-rc2, x84_64 >> > >> > Has anyone seen this before, and maybe has an idea what''s going wrong, >> > and what I need to do to get a working dom0? I''ve attached screengrabs >> > of the panic stack/call trace. >> > >> > >> > Jody >> >> Tried 2.6.30-r2 on HP ProLiant DL380G5 with xen-3.3, panics, no output on vga >> after relinquishing VGA console, works on Dell Optiplex Core2Duo. >> >> Switched back to 2.6.29-r4, works out of the box. > > Ok, built 2.6.29.6, with the v6 gentoo-xen-kernel patches, and boots fine. > So I can actually use this system again now :) > > Tried upgrading in the first place because of a bug in plain 2.6.29 that > just recently started catching me out, causing endless BUGs from linux. > Since -stable on .29 fixes it as well though, this solution is workable.What bug might that be?> > I guess the .30 patches just aren''t quite there yet, oh well.Not yet..> > > J > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Christian Fischer
2009-Aug-06 09:04 UTC
Re: [Xen-users] Xen Panic w/ dom0 2.6.30.2 (gentoo-xen-kernel.googlecode.com patches)
On Wednesday 05 August 2009, Andrew Lyon wrote:> On Wed, Aug 5, 2009 at 2:34 PM, Christian > > Fischer<Christian.Fischer@fischundfischer.com> wrote: > > On Tuesday 04 August 2009, Jody Belka wrote: > >> Hi, > >> > >> I just tried upgrading my dom0 to 2.6.30.2, using the patches on > >> gentoo-xen-kernel.googlecode.com. It built ok, but trying to boot > >> results in a "FATAL PAGE FAULT" error from xen. > >> > >> Xen specs = Xen 3.4.1-rc2, x84_64 > >> > >> Has anyone seen this before, and maybe has an idea what''s going wrong, > >> and what I need to do to get a working dom0? I''ve attached screengrabs > >> of the panic stack/call trace. > >> > >> > >> Jody > > > > Tried 2.6.30-r2 on HP ProLiant DL380G5 with xen-3.3, panics, no output on > > vga after relinquishing VGA console, works on Dell Optiplex Core2Duo. > > > > Switched back to 2.6.29-r4, works out of the box. > > > > Christian > > To be honest 2.6.29 is the best one to use at the moment, 2.6.30 has > issues with networking which I''ve not been able to fix, also opensuse > have selected 2.6.31 as their next distro kernel so I plan to rebase > that as soon as it is released and to put more effort into supporting > it, they have already moved to the 2.6.31-rcX tree so 2.6.30 is dead > as far as opensuse are concerned, that makes it much harder for me to > support. > > > AndyAndrew, dou you think 2.6.29-xen-r4 is stable enough for production use? Or is it somewhat experimental? I have "xfs issues" with 2.6.18 and Eric Sandeen recommends something more recent. Christian> > > -- > > "Without music to decorate it, time is just a bunch of boring production > > deadlines or dates by which bills must be paid." > > --- Frank Vincent Zappa > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users-- "Without music to decorate it, time is just a bunch of boring production deadlines or dates by which bills must be paid." --- Frank Vincent Zappa _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Andrew Lyon
2009-Aug-06 09:26 UTC
Re: [Xen-users] Xen Panic w/ dom0 2.6.30.2 (gentoo-xen-kernel.googlecode.com patches)
On Thu, Aug 6, 2009 at 10:04 AM, Christian Fischer<Christian.Fischer@fischundfischer.com> wrote:> On Wednesday 05 August 2009, Andrew Lyon wrote: >> On Wed, Aug 5, 2009 at 2:34 PM, Christian >> >> Fischer<Christian.Fischer@fischundfischer.com> wrote: >> > On Tuesday 04 August 2009, Jody Belka wrote: >> >> Hi, >> >> >> >> I just tried upgrading my dom0 to 2.6.30.2, using the patches on >> >> gentoo-xen-kernel.googlecode.com. It built ok, but trying to boot >> >> results in a "FATAL PAGE FAULT" error from xen. >> >> >> >> Xen specs = Xen 3.4.1-rc2, x84_64 >> >> >> >> Has anyone seen this before, and maybe has an idea what''s going wrong, >> >> and what I need to do to get a working dom0? I''ve attached screengrabs >> >> of the panic stack/call trace. >> >> >> >> >> >> Jody >> > >> > Tried 2.6.30-r2 on HP ProLiant DL380G5 with xen-3.3, panics, no output on >> > vga after relinquishing VGA console, works on Dell Optiplex Core2Duo. >> > >> > Switched back to 2.6.29-r4, works out of the box. >> > >> > Christian >> >> To be honest 2.6.29 is the best one to use at the moment, 2.6.30 has >> issues with networking which I''ve not been able to fix, also opensuse >> have selected 2.6.31 as their next distro kernel so I plan to rebase >> that as soon as it is released and to put more effort into supporting >> it, they have already moved to the 2.6.31-rcX tree so 2.6.30 is dead >> as far as opensuse are concerned, that makes it much harder for me to >> support. >> >> >> Andy > > Andrew, > > dou you think 2.6.29-xen-r4 is stable enough for production use? Or is it > somewhat experimental?Its probably the most stable Xen dom0 kernel I''ve used so far, I have it running on two production systems and I''ve had no problems. The pv_ops tree is getting better all the time but I use pci passthru and that works better with the "classic" Xen patched kernel. Just be aware that opensuse are no longer developing the patches for 2.6.29, they have already switched to 2.6.31-rcX as they have selected .31 for their next distro kernel, so if a critical bug is found and I cannot fix it or port a existing fix to .29 then we will be out of options. Andy> > I have "xfs issues" with 2.6.18 and Eric Sandeen recommends something more > recent. > > Christian >> >> > -- >> > "Without music to decorate it, time is just a bunch of boring production >> > deadlines or dates by which bills must be paid." >> > --- Frank Vincent Zappa >> > >> > _______________________________________________ >> > Xen-users mailing list >> > Xen-users@lists.xensource.com >> > http://lists.xensource.com/xen-users > > > > -- > "Without music to decorate it, time is just a bunch of boring production > deadlines or dates by which bills must be paid." > --- Frank Vincent Zappa >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Jody Belka
2009-Aug-06 11:15 UTC
Re: [Xen-users] Xen Panic w/ dom0 2.6.30.2 (gentoo-xen-kernel.googlecode.com patches)
On Thu, Aug 06, 2009 at 08:40:12AM +0100, Andrew Lyon wrote:> On Wed, Aug 5, 2009 at 11:51 PM, Jody Belka<lists-xen@pimb.org> wrote: > > Tried upgrading in the first place because of a bug in plain 2.6.29 that > > just recently started catching me out, causing endless BUGs from linux. > > Since -stable on .29 fixes it as well though, this solution is workable. > > What bug might that be?I was getting loads of bad page state BUGs all over the place. Although I don''t know for definite, I''m pretty sure it was this patch from 2.6.29.2 that fixes things... --------------------- commit 1b5479d3a60b192b9f87f63cf77772dcac2fb131 Author: NeilBrown <neilb@suse.de> Date: Mon Apr 6 20:55:03 2009 +0000 md/raid1 - don''t assume newly allocated bvecs are initialised. upstream commit: 303a0e11d0ee136ad8f53f747f3c377daece763b Since commit d3f761104b097738932afcc310fbbbbfb007ef92 newly allocated bvecs aren''t initialised to NULL, so we have to be more careful about freeing a bio which only managed to get a few pages allocated to it. Otherwise the resync process crashes. This patch is appropriate for 2.6.29-stable. Cc: stable@kernel.org Cc: "Jens Axboe" <jens.axboe@oracle.com> Reported-by: Gabriele Tozzi <gabriele@tozzi.eu> Signed-off-by: NeilBrown <neilb@suse.de> Signed-off-by: Chris Wright <chrisw@sous-sol.org> --------------------- J _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
James Dingwall
2009-Aug-06 11:18 UTC
RE: [Xen-users] Xen Panic w/ dom0 2.6.30.2(gentoo-xen-kernel.googlecode.com patches)
>> To be honest 2.6.29 is the best one to use at the moment, 2.6.30 has >> issues with networking which I''ve not been able to fix, also opensuse >> have selected 2.6.31 as their next distro kernel so I plan to rebase >> that as soon as it is released and to put more effort into supporting >> it, they have already moved to the 2.6.31-rcX tree so 2.6.30 is dead >> as far as opensuse are concerned, that makes it much harder for me to >> support.Out of interest what networking issues have you seen? I am using 2.6.30-xen-r1 and I lose connectivity to my domUs occasionally. It seems to happen under heavy disk/network I/O. I''ve not isolated to either in particular but when it seems to coincide with lots of NFS or rsync activity. e.g. my gentoo domUs are doing an rsync of the portage tree from the dom0 or doing an emerge where distfiles and the ccache are mounted by NFS from dom0. There is nothing in the logs on the domU or dom0 except from NFS timeout messages. I did try -r2 but I just get immediate reboots when xen is relinquishing the vga console. Going to reboot with 2.6.29 later to see if I still get the same behaviour. Thanks, James This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at http://www.amdocs.com/email_disclaimer.asp _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Christian Fischer
2009-Aug-06 12:48 UTC
Re: [Xen-users] Xen Panic w/ dom0 2.6.30.2 (gentoo-xen-kernel.googlecode.com patches)
On Thursday 06 August 2009, Andrew Lyon wrote:> On Thu, Aug 6, 2009 at 10:04 AM, Christian > > Fischer<Christian.Fischer@fischundfischer.com> wrote: > > On Wednesday 05 August 2009, Andrew Lyon wrote: > >> On Wed, Aug 5, 2009 at 2:34 PM, Christian > >> > >> Fischer<Christian.Fischer@fischundfischer.com> wrote: > >> > On Tuesday 04 August 2009, Jody Belka wrote: > >> >> Hi, > >> >> > >> >> I just tried upgrading my dom0 to 2.6.30.2, using the patches on > >> >> gentoo-xen-kernel.googlecode.com. It built ok, but trying to boot > >> >> results in a "FATAL PAGE FAULT" error from xen. > >> >> > >> >> Xen specs = Xen 3.4.1-rc2, x84_64 > >> >> > >> >> Has anyone seen this before, and maybe has an idea what''s going > >> >> wrong, and what I need to do to get a working dom0? I''ve attached > >> >> screengrabs of the panic stack/call trace. > >> >> > >> >> > >> >> Jody > >> > > >> > Tried 2.6.30-r2 on HP ProLiant DL380G5 with xen-3.3, panics, no output > >> > on vga after relinquishing VGA console, works on Dell Optiplex > >> > Core2Duo. > >> > > >> > Switched back to 2.6.29-r4, works out of the box. > >> > > >> > Christian > >> > >> To be honest 2.6.29 is the best one to use at the moment, 2.6.30 has > >> issues with networking which I''ve not been able to fix, also opensuse > >> have selected 2.6.31 as their next distro kernel so I plan to rebase > >> that as soon as it is released and to put more effort into supporting > >> it, they have already moved to the 2.6.31-rcX tree so 2.6.30 is dead > >> as far as opensuse are concerned, that makes it much harder for me to > >> support. > >> > >> > >> Andy > > > > Andrew, > > > > dou you think 2.6.29-xen-r4 is stable enough for production use? Or is it > > somewhat experimental? > > Its probably the most stable Xen dom0 kernel I''ve used so far, I have > it running on two production systems and I''ve had no problems.fine to hear, thx> > The pv_ops tree is getting better all the time but I use pci passthru > and that works better with the "classic" Xen patched kernel. > > Just be aware that opensuse are no longer developing the patches for > 2.6.29, they have already switched to 2.6.31-rcX as they have selected > .31 for their next distro kernel, so if a critical bug is found and I > cannot fix it or port a existing fix to .29 then we will be out of > options.Yes, you told that. I see no other solution, I''ll switch to 2.6.29. Christian> > Andy > > > I have "xfs issues" with 2.6.18 and Eric Sandeen recommends something > > more recent. > > > > Christian > > > >> > -- > >> > "Without music to decorate it, time is just a bunch of boring > >> > production deadlines or dates by which bills must be paid." > >> > --- Frank Vincent Zappa > >> > > >> > _______________________________________________ > >> > Xen-users mailing list > >> > Xen-users@lists.xensource.com > >> > http://lists.xensource.com/xen-users > > > > -- > > "Without music to decorate it, time is just a bunch of boring production > > deadlines or dates by which bills must be paid." > > --- Frank Vincent Zappa-- "Without music to decorate it, time is just a bunch of boring production deadlines or dates by which bills must be paid." --- Frank Vincent Zappa _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Andrew Lyon
2009-Aug-07 07:29 UTC
Re: [Xen-users] Xen Panic w/ dom0 2.6.30.2(gentoo-xen-kernel.googlecode.com patches)
On Thu, Aug 6, 2009 at 12:18 PM, James Dingwall<james.dingwall@amdocs.com> wrote:> >>> To be honest 2.6.29 is the best one to use at the moment, 2.6.30 has >>> issues with networking which I''ve not been able to fix, also opensuse >>> have selected 2.6.31 as their next distro kernel so I plan to rebase >>> that as soon as it is released and to put more effort into supporting >>> it, they have already moved to the 2.6.31-rcX tree so 2.6.30 is dead >>> as far as opensuse are concerned, that makes it much harder for me to >>> support. > > Out of interest what networking issues have you seen? I am using > 2.6.30-xen-r1 and I lose connectivity to my domUs occasionally. It > seems to happen under heavy disk/network I/O. I''ve not isolated to > either in particular but when it seems to coincide with lots of NFS or > rsync activity. e.g. my gentoo domUs are doing an rsync of the portage > tree from the dom0 or doing an emerge where distfiles and the ccache are > mounted by NFS from dom0. There is nothing in the logs on the domU or > dom0 except from NFS timeout messages. I did try -r2 but I just get > immediate reboots when xen is relinquishing the vga console. Going to > reboot with 2.6.29 later to see if I still get the same behaviour. > > Thanks, > James > > This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, > you may review at http://www.amdocs.com/email_disclaimer.asp >The networking problems I''ve had with 2.6.30 are malformed packets being sent to the domU, see the last few posts in this thread for details http://forums.gentoo.org/viewtopic.php?p=5915696#5915696 Andy _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
James Dingwall
2009-Aug-27 09:33 UTC
RE: [Xen-users] Xen Panic w/ dom0 2.6.30.2(gentoo-xen-kernel.googlecode.com patches)
> On Thu, Aug 6, 2009 at 12:18 PM, James > Dingwall<james.dingwall@amdocs.com> wrote: > > > >>> To be honest 2.6.29 is the best one to use at the moment, 2.6.30 > has > >>> issues with networking which I''ve not been able to fix, also > opensuse > >>> have selected 2.6.31 as their next distro kernel so I plan to > rebase > >>> that as soon as it is released and to put more effort into > supporting > >>> it, they have already moved to the 2.6.31-rcX tree so 2.6.30 is > dead > >>> as far as opensuse are concerned, that makes it much harder for me > to > >>> support. > > > > Out of interest what networking issues have you seen? I am using > > 2.6.30-xen-r1 and I lose connectivity to my domUs occasionally. It > > seems to happen under heavy disk/network I/O. I''ve not isolated to > > either in particular but when it seems to coincide with lots of NFS > or > > rsync activity. e.g. my gentoo domUs are doing an rsync of the > portage > > tree from the dom0 or doing an emerge where distfiles and the ccache > are > > mounted by NFS from dom0. There is nothing in the logs on the domU > or > > dom0 except from NFS timeout messages. I did try -r2 but I just get > > immediate reboots when xen is relinquishing the vga console. Going > to > > reboot with 2.6.29 later to see if I still get the same behaviour. > > > > Thanks, > > James > > > > This message and the information contained herein is proprietary and > confidential and subject to the Amdocs policy statement, > > you may review at http://www.amdocs.com/email_disclaimer.asp > > > > The networking problems I''ve had with 2.6.30 are malformed packets > being sent to the domU, see the last few posts in this thread for > details http://forums.gentoo.org/viewtopic.php?p=5915696#5915696FWIW I seem to have solved the network problems I was experiencing by changing the following .config options in the kernel: # CONFIG_XEN_COMPAT_030002_AND_LATER is not set # CONFIG_XEN_COMPAT_030004_AND_LATER is not set # CONFIG_XEN_COMPAT_030100_AND_LATER is not set # CONFIG_XEN_COMPAT_030200_AND_LATER is not set # CONFIG_XEN_COMPAT_030300_AND_LATER is not set CONFIG_XEN_COMPAT_LATEST_ONLY=y (Was CONFIG_XEN_COMPAT_030002_AND_LATER=y) CONFIG_XEN_NETDEV_TX_SHIFT=10 (Was CONFIG_XEN_NETDEV_TX_SHIFT=8) James This message and the information contained herein is proprietary and confidential and subject to the Amdocs policy statement, you may review at http://www.amdocs.com/email_disclaimer.asp _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users