On Mon, Jan 02, 2017 at 06:57:48AM -0500, Aryeh Friedman wrote:> FreeBSD lilith 11.0-STABLE FreeBSD 11.0-STABLE #7 r311003: Sun Jan 1 > 02:45:34 EST 2017 root at lilith:/usr/obj/usr/src/sys/GENERIC amd64 > > > -------------------------------------------------------------- > >>> stage 3.1: building everything > -------------------------------------------------------------- > cd /usr/obj/usr/src/sys/GENERIC; COMPILER_VERSION=30901 > COMPILER_TYPE=clang COMPILER_FREEBSD_VERSION=1100503 > MAKEOBJDIRPREFIX=/usr/obj MACHINE_ARCH=amd64 MACHINE=amd64 CPUTYPE> GROFF_BIN_PATH=/usr/obj/usr/src/tmp/legacy/usr/bin > GROFF_FONT_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/groff_font > GROFF_TMAC_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/tmac CC="cc -target > x86_64-unknown-freebsd11.0 --sysroot=/usr/obj/usr/src/tmp > -B/usr/obj/usr/src/tmp/usr/bin" CXX="c++ -target > x86_64-unknown-freebsd11.0 --sysroot=/usr/obj/usr/src/tmp > -B/usr/obj/usr/src/tmp/usr/bin" CPP="cpp -target > x86_64-unknown-freebsd11.0 --sysroot=/usr/obj/usr/src/tmp > -B/usr/obj/usr/src/tmp/usr/bin" AS="as" AR="ar" LD="ld" NM=nm > OBJDUMP=objdump OBJCOPY="objcopy" RANLIB=ranlib STRINGS= SIZE="size" > INSTALL="sh /usr/src/tools/install.sh" > PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/bin:/usr/obj/usr/src/tmp/usr/sbin:/usr/obj/usr/src/tmp/usr/bin:/sbin:/bin:/usr/sbin:/usr/bin > make -m /usr/src/share/mk KERNEL=kernel all -DNO_MODULES_OBJ > linking kernel.full > ctfmerge -L VERSION -g -o kernel.full ... > <freezes>How reproducible is the crash? What previous kernel was known to work? Can you narrow it down to a particular revision, preferably with kernel debugging enabled? (see the end of the mail) There was one invasive change merged - fine-grained namecache in r310959 and that can be treated as the likely culprit. That said, I would start the search with verifying there are no issues with r310953 first. Debug opts: options KDB # Enable kernel debugger support. options KDB_TRACE # Print a stack trace for a panic. # For full debugger support use (turn off in stable branch): options DDB # Support DDB. options GDB # Support remote GDB. options INVARIANTS # Enable calls of extra sanity checking options INVARIANT_SUPPORT # Extra sanity checks of internal structures, required by INVARIANTS options WITNESS # Enable checks to detect deadlocks and cycles options WITNESS_SKIPSPIN # Don't run witness on spinlocks for speed options MALLOC_DEBUG_MAXZONES=8 # Separate malloc(9) zones options DEBUG_VFS_LOCKS -- Mateusz Guzik <mjguzik gmail.com>
On Mon, Jan 02, 2017 at 01:36:31PM +0100, Mateusz Guzik wrote:> On Mon, Jan 02, 2017 at 06:57:48AM -0500, Aryeh Friedman wrote: > > FreeBSD lilith 11.0-STABLE FreeBSD 11.0-STABLE #7 r311003: Sun Jan 1 > > 02:45:34 EST 2017 root at lilith:/usr/obj/usr/src/sys/GENERIC amd64 > > ... > > make -m /usr/src/share/mk KERNEL=kernel all -DNO_MODULES_OBJ > > linking kernel.full > > ctfmerge -L VERSION -g -o kernel.full ... > > <freezes> > > How reproducible is the crash? What previous kernel was known to work? > Can you narrow it down to a particular revision, preferably with kernel > debugging enabled? (see the end of the mail)FWIW, I did not see anything approaching such a freeze, either on my build machine or my laptop, during the just-comopleted upgrade from: FreeBSD g1-252.catwhisker.org 11.0-STABLE FreeBSD 11.0-STABLE #209 r311007M/311007:1100508: Sun Jan 1 03:51:25 PST 2017 root at g1-252.catwhisker.org:/common/S1/obj/usr/src/sys/CANARY amd64 to: FreeBSD g1-252.catwhisker.org 11.0-STABLE FreeBSD 11.0-STABLE #210 r311047M/311097:1100508: Mon Jan 2 04:23:25 PST 2017 root at g1-252.catwhisker.org:/common/S1/obj/usr/src/sys/CANARY amd64 (Or any prior upgrade, that I recall).> ....Peace, david -- David H. Wolfskill david at catwhisker.org Epistemology for post-truthers: How do we select parts of reality to ignore? See http://www.catwhisker.org/~david/publickey.gpg for my public key. -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 603 bytes Desc: not available URL: <http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20170102/13dacb05/attachment.sig>
On Mon, Jan 2, 2017 at 7:36 AM, Mateusz Guzik <mjguzik at gmail.com> wrote:> On Mon, Jan 02, 2017 at 06:57:48AM -0500, Aryeh Friedman wrote: > > FreeBSD lilith 11.0-STABLE FreeBSD 11.0-STABLE #7 r311003: Sun Jan 1 > > 02:45:34 EST 2017 root at lilith:/usr/obj/usr/src/sys/GENERIC amd64 > > > > > > -------------------------------------------------------------- > > >>> stage 3.1: building everything > > -------------------------------------------------------------- > > cd /usr/obj/usr/src/sys/GENERIC; COMPILER_VERSION=30901 > > COMPILER_TYPE=clang COMPILER_FREEBSD_VERSION=1100503 > > MAKEOBJDIRPREFIX=/usr/obj MACHINE_ARCH=amd64 MACHINE=amd64 CPUTYPE> > GROFF_BIN_PATH=/usr/obj/usr/src/tmp/legacy/usr/bin > > GROFF_FONT_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/groff_font > > GROFF_TMAC_PATH=/usr/obj/usr/src/tmp/legacy/usr/share/tmac CC="cc > -target > > x86_64-unknown-freebsd11.0 --sysroot=/usr/obj/usr/src/tmp > > -B/usr/obj/usr/src/tmp/usr/bin" CXX="c++ -target > > x86_64-unknown-freebsd11.0 --sysroot=/usr/obj/usr/src/tmp > > -B/usr/obj/usr/src/tmp/usr/bin" CPP="cpp -target > > x86_64-unknown-freebsd11.0 --sysroot=/usr/obj/usr/src/tmp > > -B/usr/obj/usr/src/tmp/usr/bin" AS="as" AR="ar" LD="ld" NM=nm > > OBJDUMP=objdump OBJCOPY="objcopy" RANLIB=ranlib STRINGS= SIZE="size" > > INSTALL="sh /usr/src/tools/install.sh" > > PATH=/usr/obj/usr/src/tmp/legacy/usr/sbin:/usr/obj/usr/ > src/tmp/legacy/usr/bin:/usr/obj/usr/src/tmp/legacy/bin:/ > usr/obj/usr/src/tmp/usr/sbin:/usr/obj/usr/src/tmp/usr/bin:/ > sbin:/bin:/usr/sbin:/usr/bin > > make -m /usr/src/share/mk KERNEL=kernel all -DNO_MODULES_OBJ > > linking kernel.full > > ctfmerge -L VERSION -g -o kernel.full ... > > <freezes> > > How reproducible is the crash? What previous kernel was known to work? > Can you narrow it down to a particular revision, preferably with kernel > debugging enabled? (see the end of the mail) >It first appeared a few days ago (forget what revision) then disappeared the day after and reappeared yesterday. It is 100% reproducible (i.e. clearing out /usr/obj and doing a make kernel in either single or multiuser mode both cause it). Turing on debugging would be hard but perhaps I should slightly qualify "freeze": make freezes but the rest of the system is responsive and killing make leaves a zombie ctfmerge. If I still need kernel debugging based on the above I will do it but looking for an easier explanation first.> >-- Aryeh M. Friedman, Lead Developer, http://www.PetiteCloud.org