Marc G. Fournier
2005-Jul-29 02:21 UTC
Consistent file system hang with RELENG_6 of today ...
'k, I'm starting to play with 6.x, for our new server ... my priority right now is to just have it run the existing 'jail' environments from my 4.x machine, while I work on getting all of our servers up to 6.x, and then will worry about the jail's themselves ... When I try and startup the 4.x jail on my 6.x machine, it "hangs" the file system that the jail directory hierarchy happens to be mounted on though ... twice in a row so far ... Now, I'm suspecting (and am going to try without it) that it might be because I'm mounting devfs within the 4.x jail, but even then, it shouldn't hang things up, only generate a whack of errors ... I have a good dump (CTL-ALT-ESC -> panic), but do not have a clue what to offer from within there that might be of any use ... If anyone is interested ... ? thanks ... ---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
On 7/28/05, Marc G. Fournier <scrappy@hub.org> wrote:> When I try and startup the 4.x jail on my 6.x machine, it "hangs" the file > system that the jail directory hierarchy happens to be mounted on though > ... twice in a row so far ... >Do you have COMPAT_FREEBSD4 in the 6.x kernel config? This might be the cause of the hangs, as it can't find the FreeBSD4 compatibilty code in the kernel. -- DISCLAIMER: No electrons were mamed while sending this message. Only slightly bruised.
Marc G. Fournier
2005-Jul-29 03:22 UTC
Consistent file system hang with RELENG_6 of today ...
And now, to follow up on my original ... I now have two core dumps, if anyone is interested :) To give some details as to what I'm trying to do here, and how ... I have a RELENG_6 machine, updated this afternoon, that I am trying to run a 4.x based jail environment on ... and these are what are live/running on my 6 productions 4.x machines right now ... I have COMPAT_FREEBSD4 built into the kernel:> strings /boot/kernel/kernel | grep -i COMPAT_ | grep __options___options COMPAT_43 #Compatible with BSD 4.3 [KEEP THIS!] ___options COMPAT_FREEBSD4 Now, as most of the "old timers" here know, my jail environment makes use of unionfs to "mount" a template layer, to share common binaries amongst the VMs ... I have tried two different ways of doing this, both result in the exact same 'file system hang' ... and I have a core of each 'method' ... The first was to use mount_devfs to, of course, mount the /dev directory within the jail itself ... The second, I used the same /dev that existed within the jail, based on bulding a jail using a 4.x system ... In both cases, the hang appears to be at the same spot, where 'sendmail' (in this case, the postfix port) starts up ... In both cases, I was able to enter DDB using CTL-ALT-ESC and create a dump, but, unlike a server crash, where you just basically bt the core file, I really haven't got a clue where to even start to provide information from the core's themselves ... So, is there anyone out there able/willing to provide some direction on providing more information? Thanks ... On Thu, 28 Jul 2005, Marc G. Fournier wrote:> > 'k, I'm starting to play with 6.x, for our new server ... my priority right > now is to just have it run the existing 'jail' environments from my 4.x > machine, while I work on getting all of our servers up to 6.x, and then will > worry about the jail's themselves ... > > When I try and startup the 4.x jail on my 6.x machine, it "hangs" the file > system that the jail directory hierarchy happens to be mounted on though ... > twice in a row so far ... > > Now, I'm suspecting (and am going to try without it) that it might be because > I'm mounting devfs within the 4.x jail, but even then, it shouldn't hang > things up, only generate a whack of errors ... > > I have a good dump (CTL-ALT-ESC -> panic), but do not have a clue what to > offer from within there that might be of any use ... > > If anyone is interested ... ? > > thanks ... > > ---- > Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) > Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664 >---- Marc G. Fournier Hub.Org Networking Services (http://www.hub.org) Email: scrappy@hub.org Yahoo!: yscrappy ICQ: 7615664
Robert Watson
2005-Jul-29 08:24 UTC
Consistent file system hang with RELENG_6 of today ...
On Thu, 28 Jul 2005, Marc G. Fournier wrote:> 'k, I'm starting to play with 6.x, for our new server ... my priority > right now is to just have it run the existing 'jail' environments from > my 4.x machine, while I work on getting all of our servers up to 6.x, > and then will worry about the jail's themselves ... > > When I try and startup the 4.x jail on my 6.x machine, it "hangs" the > file system that the jail directory hierarchy happens to be mounted on > though ... twice in a row so far ... > > Now, I'm suspecting (and am going to try without it) that it might be > because I'm mounting devfs within the 4.x jail, but even then, it > shouldn't hang things up, only generate a whack of errors ... > > I have a good dump (CTL-ALT-ESC -> panic), but do not have a clue what > to offer from within there that might be of any use ... > > If anyone is interested ... ?If you can get into DDB and have serial console output, the following would be useful: The output of 'show pcpu' The output of 'show pcpu X' for each present cpu, starting with 0. The output of 'ps' The output of 'trace' for the currently running thread, and each non-idle thread shown in the show pcpu output The output of 'show lockedvnods' It would also be useful if, relating to the startup of the jail, you can identify the point in the jail boot where it wedges, and if you hit Ctrl-T, what process is shown as running and what state it is in, and using DDB, trace that process. If you could show the trace output for each process listed in "show lockevnods". Likely, there is a leaked lock or a low buffer condition. However, once we have the above output we should be able to say more. The above will hopefully tell us whether it's a vnode deadlock, and ideally, the approximate source. Robert N M Watson