freebsd stable - Feb 2006 - panic: thread with borrowed priority returning to userland

Panic on 6.1-PRERELEASE (package build machine):

db> wh
Tracing pid 85657 tid 100133 td 0xc2612600
kdb_enter(c06cabdd,c07250c0,c06cc327,cdca4cac,100) at kdb_enter+0x30
panic(c06cc327,2,c06cc3bf,267,2814d4e8) at panic+0xd5
sched_userret(c2612600,c04e290d,c0724dc0,1,2814d4e8) at sched_userret+0x21
userret(c2612600,cdca4d38,63,100,1010000) at userret+0xd8
ast(cdca4d38) at ast+0x556
doreti_ast() at doreti_ast+0x17

Core available.

Kris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url :
http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20060225/9a2ed5f4/attachment.bin

On Sat, Feb 25, 2006 at 09:17:30AM -0500, Kris Kennaway
wrote:
> Panic on 6.1-PRERELEASE (package build machine):
> 
> db> wh
> Tracing pid 85657 tid 100133 td 0xc2612600
> kdb_enter(c06cabdd,c07250c0,c06cc327,cdca4cac,100) at kdb_enter+0x30
> panic(c06cc327,2,c06cc3bf,267,2814d4e8) at panic+0xd5
> sched_userret(c2612600,c04e290d,c0724dc0,1,2814d4e8) at sched_userret+0x21
> userret(c2612600,cdca4d38,63,100,1010000) at userret+0xd8
> ast(cdca4d38) at ast+0x556
> doreti_ast() at doreti_ast+0x17
> 
> Core available.
The only other process active was:

85837 c239d000    0 85836 73099 0004000 [LOCK vnode interlock c23cae40] pkg_add

which is holding a lock:

db> show lockedvnods
Locked vnodes

0xc2d09000: tag devfs, type VDIR
    usecount 2, writecount 0, refcount 2 mountedhere 0
    flags (VV_ROOT)
     lock type devfs: EXCL (count 1) by thread 0xc1f1fc00 (pid 85837)
db> wh 85837
Tracing pid 85837 tid 100077 td 0xc1f1fc00
sched_switch(c1f1fc00,0,1,118,8948ac5f) at sched_switch+0x190
mi_switch(1,0,c06cdf0d,253,c07293a8) at mi_switch+0x2e6
turnstile_wait(c3192d3c,c2612600,c06c9e64,225,c2612602) at turnstile_wait+0x3b6
_mtx_lock_sleep(c3192d3c,c1f1fc00,0,c06d264c,797) at _mtx_lock_sleep+0x137
_mtx_lock_flags(c3192d3c,0,c06d264c,797,2) at _mtx_lock_flags+0x9b
vget(c3192cc0,2,c1f1fc00,c23e8405,c2a2ce60) at vget+0x47
devfs_allocv(c2a2ce00,c1f5c400,cc9e8bd8,c1f1fc00,cc9e88a8) at devfs_allocv+0x43
devfs_lookupx(cc9e8998,c06c3419,23f,c06f89c0,cc9e8998) at devfs_lookupx+0x569
devfs_lookup(cc9e8998,cc9e8998,c2d09000,c2d09000,0) at devfs_lookup+0x3e
VOP_LOOKUP_APV(c06f89c0,cc9e8998,cc9e8984,c1f1fc00,0) at VOP_LOOKUP_APV+0xb4
lookup(cc9e8bc4,0,c06d1dd1,b6,1b2) at lookup+0x488
namei(cc9e8bc4,c33bf660,c1f1fc00,cc9e8a58,c05529ed) at namei+0x468
vn_open_cred(cc9e8bc4,cc9e8cc4,0,c2c5e700,3) at vn_open_cred+0x2d8
vn_open(cc9e8bc4,cc9e8cc4,0,3,0) at vn_open+0x33
kern_open(c1f1fc00,2827b19f,0,1,0) at kern_open+0xc8
open(c1f1fc00,cc9e8d04,c,41d,3) at open+0x36
syscall(3b,3b,3b,bfbfd6f0,bfbfd705) at syscall+0x2c0
Xint0x80_syscall() at Xint0x80_syscall+0x1f
--- syscall (5, FreeBSD ELF32, open), eip = 0x28264eab, esp = 0xbfbfb7bc, ebp =
0xbfbfb8b8 ---
db>

Interestingly, almost the same trace appeared in an amd64 7.0 panic
overnight, on a machine with DEBUG_VFS_LOCKS enabled:

VNASSERT failed
0xffffff03067c13b0: tag none, type VBAD
    usecount 0, writecount 0, refcount 1 mountedhere 0
    flags (VI_DOOMED)
 VI_LOCKed
panic: No vop_lock(0xffffff03067c13b0, 0xffffffffae46c330)
cpuid = 3
KDB: enter: panic
[thread pid 5883 tid 100224 ]
Stopped at      kdb_enter+0x31: leave
db> wh
Tracing pid 5883 tid 100224 td 0xffffff0310aa97e0
kdb_enter() at kdb_enter+0x31
panic() at panic+0x1e6
VOP_LOCK_APV() at VOP_LOCK_APV+0xb4
vn_lock() at vn_lock+0xc8
vget() at vget+0xf0
devfs_allocv() at devfs_allocv+0x60
devfs_lookupx() at devfs_lookupx+0x53b
devfs_lookup() at devfs_lookup+0x43
VOP_LOOKUP_APV() at VOP_LOOKUP_APV+0xbb
lookup() at lookup+0x4b5
namei() at namei+0x3fb
vn_open_cred() at vn_open_cred+0x95
vn_open() at vn_open+0x17
kern_open() at kern_open+0xe6
open() at open+0x28
syscall() at syscall+0x350
Xfast_syscall() at Xfast_syscall+0xa8
--- syscall (5, FreeBSD ELF64, open), rip = 0x800979dec, rsp = 0x7fffffffc4e8,
rbp = 0x5c2dd0 ---
db>

Kris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url :
http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20060225/e9f4eb7e/attachment.bin

On Saturday 25 February 2006 09:17, Kris Kennaway wrote:
> Panic on 6.1-PRERELEASE (package build machine):
> 
> db> wh
> Tracing pid 85657 tid 100133 td 0xc2612600
> kdb_enter(c06cabdd,c07250c0,c06cc327,cdca4cac,100) at kdb_enter+0x30
> panic(c06cc327,2,c06cc3bf,267,2814d4e8) at panic+0xd5
> sched_userret(c2612600,c04e290d,c0724dc0,1,2814d4e8) at sched_userret+0x21
> userret(c2612600,cdca4d38,63,100,1010000) at userret+0xd8
> ast(cdca4d38) at ast+0x556
> doreti_ast() at doreti_ast+0x17
> 
> Core available.
Does this thread (tid 100133) hold any locks?  That panic would
be indicative of that since it drops its inherited priority when it
releaes a contested mutex.

-- 
John Baldwin <jhb@FreeBSD.org>  <><  http://www.FreeBSD.org/~jhb/
"Power Users Use the Power to Serve"  =  http://www.FreeBSD.org

On Tue, Feb 28, 2006 at 12:07:37PM -0500, John Baldwin
wrote:
> On Saturday 25 February 2006 09:17, Kris Kennaway wrote:
> > Panic on 6.1-PRERELEASE (package build machine):
> > 
> > db> wh
> > Tracing pid 85657 tid 100133 td 0xc2612600
> > kdb_enter(c06cabdd,c07250c0,c06cc327,cdca4cac,100) at kdb_enter+0x30
> > panic(c06cc327,2,c06cc3bf,267,2814d4e8) at panic+0xd5
> > sched_userret(c2612600,c04e290d,c0724dc0,1,2814d4e8) at
sched_userret+0x21
> > userret(c2612600,cdca4d38,63,100,1010000) at userret+0xd8
> > ast(cdca4d38) at ast+0x556
> > doreti_ast() at doreti_ast+0x17
> > 
> > Core available.
> 
> Does this thread (tid 100133) hold any locks?  That panic would
> be indicative of that since it drops its inherited priority when it
> releaes a contested mutex.
See other followups for diagnosis.

Kris
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: not available
Url :
http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20060228/5a29807e/attachment.bin