Hello!
I had some troubles mounting the filesystem from:
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <MATSHITA DMC-FX12 0100> Removable Direct Access SCSI-2 device
da0: 1.000MB/s transfers
da0: 3886MB (7959552 512 byte sectors: 255H 63S/T 495C)
and decided to just dd the entire da0 to a file, so that the camera
can be disconnected:
dd if=/dev/da0 of=/home/mi/da0.dd bs=16384
The dd-ing was proceeding slowly (600Kb/s) and I stopped watching it...
The machine paniced about an hour later (at 0:52). The timestamp on
/home/mi/da0.dd was 23:45, it was only about 500Mb in size.
The stack is below. Would anybody like to look at the complete
vmcore dump?
The hardware is a quad Opteron with 8Gb RAM. Only 4Gb of these are
used, because it runs 7.x/i386 from April 5th (without PAE) -- for
the sake of NVidia's card.
Please, advise. Thanks!
-mi
[GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so:
Undefined symbol "ps_pglobal_lookup"]
GNU gdb 6.1.1 [FreeBSD]
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB. Type "show warranty" for
details.
This GDB was configured as "i386-marcel-freebsd".
There is no member named pathname.
Reading symbols from /boot/modules/nvidia.ko...done.
Loaded symbols for /boot/modules/nvidia.ko
Reading symbols from /opt/modules/fuse.ko...done.
Loaded symbols for /opt/modules/fuse.ko
Unread portion of the kernel message buffer:
umass0: at uhub0 port 6 (addr 2) disconnected
(da0:umass-sim0:0:0:0): lost device
Fatal trap 12: page fault while in kernel mode
cpuid = 3; apic id = 03
fault virtual address = 0x0
fault code = supervisor write, page not present
instruction pointer = 0x20:0xc0449702
stack pointer = 0x28:0xeb74b8bc
frame pointer = 0x28:0xeb74b8dc
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = interrupt enabled, resume, IOPL = 0
current process = 13989 (dd)
trap number = 12
panic: page fault
cpuid = 3
Uptime: 12d10h52m16s
(da0:dead_sim0:0:0:0): Synchronize cache failed, status == 0x34, scsi status ==
0xc8
Physical memory: 3054 MB
Dumping 334 MB: (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C
to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to
abort) 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31
15
#0 doadump () at pcpu.h:195
195 __asm __volatile("movl %%fs:0,%0" : "=r" (td));
(kgdb) #0 doadump () at pcpu.h:195
#1 0xc0599f7b in boot (howto=260) at /ibm/src/sys/kern/kern_shutdown.c:418
#2 0xc059a449 in panic (fmt=0x104 <Address 0x104 out of bounds>)
at /ibm/src/sys/kern/kern_shutdown.c:572
#3 0xc077f60d in trap_fatal (frame=0xeb74b87c, eva=40)
at /ibm/src/sys/i386/i386/trap.c:899
#4 0xc077f9aa in trap_pfault (frame=0xeb74b87c, usermode=0, eva=0)
at /ibm/src/sys/i386/i386/trap.c:812
#5 0xc078035c in trap (frame=0xeb74b87c) at /ibm/src/sys/i386/i386/trap.c:490
#6 0xc076637b in calltrap () at /ibm/src/sys/i386/i386/exception.s:139
#7 0xc0449702 in xpt_done (done_ccb=0xc690a000)
at /ibm/src/sys/cam/cam_xpt.c:4856
#8 0xc044b15c in xpt_action (start_ccb=0xc690a000)
at /ibm/src/sys/cam/cam_xpt.c:3057
#9 0xc04462b6 in cam_periph_runccb (ccb=0xc690a000, error_routine=0,
camflags=CAM_FLAG_NONE, sense_flags=1, ds=0xc6aea690)
at /ibm/src/sys/cam/cam_periph.c:878
#10 0xc0453aa1 in daclose (dp=0xcc862600)
at /ibm/src/sys/cam/scsi/scsi_da.c:714
#11 0xc0549b2e in g_disk_access (pp=0xc7e12680, r=0, w=0, e=Variable
"e" is not available.)
at /ibm/src/sys/geom/geom_disk.c:152
#12 0xc054ec4d in g_access (cp=0xc8a90380, dcr=-1, dcw=0, dce=0)
at /ibm/src/sys/geom/geom_subr.c:748
#13 0xc05490f3 in g_dev_close (dev=0xca1dad00, flags=Variable "flags"
is not available.)
at /ibm/src/sys/geom/geom_dev.c:217
#14 0xc0531f69 in devfs_close (ap=0xeb74ba94)
at /ibm/src/sys/fs/devfs/devfs_vnops.c:372
#15 0xc0623e86 in vn_close (vp=0xcc8c1dd0, flags=1, file_cred=0xca36e700,
td=0xc8552000) at vnode_if.h:228
#16 0xc0623f37 in vn_closefile (fp=0xca19bdc8, td=0xc8552000)
at /ibm/src/sys/kern/vfs_vnops.c:868
#17 0xc0568b14 in fdrop (fp=0xca19bdc8, td=0xc8552000) at file.h:297
#18 0xc056a3c2 in closef (fp=0xca19bdc8, td=0xc8552000)
at /ibm/src/sys/kern/kern_descrip.c:1958
#19 0xc056b371 in fdfree (td=0xc8552000)
at /ibm/src/sys/kern/kern_descrip.c:1668
#20 0xc05775fa in exit1 (td=0xc8552000, rv=256)
at /ibm/src/sys/kern/kern_exit.c:272
#21 0xc0578e1d in sys_exit (td=Could not find the frame base for
"sys_exit".
) at /ibm/src/sys/kern/kern_exit.c:98
#22 0xc077fb90 in syscall (frame=0xeb74bd38)
at /ibm/src/sys/i386/i386/trap.c:1035
#23 0xc07663e0 in Xint0x80_syscall () at /ibm/src/sys/i386/i386/exception.s:196
#24 0x00000033 in ?? ()
Previous frame inner to this frame (corrupt stack?)
(kgdb)
Arno J. Klaassen
2008-Jun-06 17:37 UTC
[nvidia | shared irq] umass disconnects [was: panic dd-ing from a USB "disk" ]
Mikhail Teterin <mi@symbion.zaytman.com> writes:> Hello! > > I had some troubles mounting the filesystem from: > > da0 at umass-sim0 bus 0 target 0 lun 0 > da0: <MATSHITA DMC-FX12 0100> Removable Direct Access SCSI-2 device > da0: 1.000MB/s transfers > da0: 3886MB (7959552 512 byte sectors: 255H 63S/T 495C) > > and decided to just dd the entire da0 to a file, so that the camera > can be disconnected: > > dd if=/dev/da0 of=/home/mi/da0.dd bs=16384 > > The dd-ing was proceeding slowly (600Kb/s) and I stopped watching it... > > The machine paniced about an hour later (at 0:52). The timestamp on > /home/mi/da0.dd was 23:45, it was only about 500Mb in size. > > The stack is below. Would anybody like to look at the complete > vmcore dump? > > The hardware is a quad Opteron with 8Gb RAM. Only 4Gb of these are > used, because it runs 7.x/i386 from April 5th (without PAE) -- for > the sake of NVidia's card.I can easily produce a similar panic on a dual Opteron 185 with 3G of RAM and running 7-stable-amd64 on a (cheap) nvidia-based MB. It runs gmirror on atapci1 and I attach a geli-encrypted disk via usb. Both share irq 23. Under heavy load ("periodic security" is enough ) it panics after having disconnected umass0 ( kgdb trace below ) : Unread portion of the kernel message buffer: umass0: at uhub1 port 1 (addr 2) disconnected (da1:umass-sim0:0:0:0): lost device (pass1:umass-sim0:0:0:0): lost device (pass1:umass-sim0:0:0:0): removing device entry I'd be happy to provide more info. Best, Arno> Please, advise. Thanks! > > -mi > > [GDB will not be able to debug user-mode threads: /usr/lib/libthread_db.so: Undefined symbol "ps_pglobal_lookup"] > GNU gdb 6.1.1 [FreeBSD] > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "i386-marcel-freebsd". > There is no member named pathname. > Reading symbols from /boot/modules/nvidia.ko...done. > Loaded symbols for /boot/modules/nvidia.ko > Reading symbols from /opt/modules/fuse.ko...done. > Loaded symbols for /opt/modules/fuse.ko > > Unread portion of the kernel message buffer: > umass0: at uhub0 port 6 (addr 2) disconnected > (da0:umass-sim0:0:0:0): lost device > > > Fatal trap 12: page fault while in kernel mode > cpuid = 3; apic id = 03 > fault virtual address = 0x0 > fault code = supervisor write, page not present > instruction pointer = 0x20:0xc0449702 > stack pointer = 0x28:0xeb74b8bc > frame pointer = 0x28:0xeb74b8dc > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = interrupt enabled, resume, IOPL = 0 > current process = 13989 (dd) > trap number = 12 > panic: page fault > cpuid = 3 > Uptime: 12d10h52m16s > (da0:dead_sim0:0:0:0): Synchronize cache failed, status == 0x34, scsi status == 0xc8 > Physical memory: 3054 MB > Dumping 334 MB: (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) (CTRL-C to abort) 319 303 287 271 255 239 223 207 191 175 159 143 127 111 95 79 63 47 31 15 > > #0 doadump () at pcpu.h:195 > 195 __asm __volatile("movl %%fs:0,%0" : "=r" (td)); > (kgdb) #0 doadump () at pcpu.h:195 > #1 0xc0599f7b in boot (howto=260) at /ibm/src/sys/kern/kern_shutdown.c:418 > #2 0xc059a449 in panic (fmt=0x104 <Address 0x104 out of bounds>) > at /ibm/src/sys/kern/kern_shutdown.c:572 > #3 0xc077f60d in trap_fatal (frame=0xeb74b87c, eva=40) > at /ibm/src/sys/i386/i386/trap.c:899 > #4 0xc077f9aa in trap_pfault (frame=0xeb74b87c, usermode=0, eva=0) > at /ibm/src/sys/i386/i386/trap.c:812 > #5 0xc078035c in trap (frame=0xeb74b87c) at /ibm/src/sys/i386/i386/trap.c:490 > #6 0xc076637b in calltrap () at /ibm/src/sys/i386/i386/exception.s:139 > #7 0xc0449702 in xpt_done (done_ccb=0xc690a000) > at /ibm/src/sys/cam/cam_xpt.c:4856 > #8 0xc044b15c in xpt_action (start_ccb=0xc690a000) > at /ibm/src/sys/cam/cam_xpt.c:3057 > #9 0xc04462b6 in cam_periph_runccb (ccb=0xc690a000, error_routine=0, > camflags=CAM_FLAG_NONE, sense_flags=1, ds=0xc6aea690) > at /ibm/src/sys/cam/cam_periph.c:878 > #10 0xc0453aa1 in daclose (dp=0xcc862600) > at /ibm/src/sys/cam/scsi/scsi_da.c:714 > #11 0xc0549b2e in g_disk_access (pp=0xc7e12680, r=0, w=0, e=Variable "e" is not available.) > at /ibm/src/sys/geom/geom_disk.c:152 > #12 0xc054ec4d in g_access (cp=0xc8a90380, dcr=-1, dcw=0, dce=0) > at /ibm/src/sys/geom/geom_subr.c:748 > #13 0xc05490f3 in g_dev_close (dev=0xca1dad00, flags=Variable "flags" is not available.) > at /ibm/src/sys/geom/geom_dev.c:217 > #14 0xc0531f69 in devfs_close (ap=0xeb74ba94) > at /ibm/src/sys/fs/devfs/devfs_vnops.c:372 > #15 0xc0623e86 in vn_close (vp=0xcc8c1dd0, flags=1, file_cred=0xca36e700, > td=0xc8552000) at vnode_if.h:228 > #16 0xc0623f37 in vn_closefile (fp=0xca19bdc8, td=0xc8552000) > at /ibm/src/sys/kern/vfs_vnops.c:868 > #17 0xc0568b14 in fdrop (fp=0xca19bdc8, td=0xc8552000) at file.h:297 > #18 0xc056a3c2 in closef (fp=0xca19bdc8, td=0xc8552000) > at /ibm/src/sys/kern/kern_descrip.c:1958 > #19 0xc056b371 in fdfree (td=0xc8552000) > at /ibm/src/sys/kern/kern_descrip.c:1668 > #20 0xc05775fa in exit1 (td=0xc8552000, rv=256) > at /ibm/src/sys/kern/kern_exit.c:272 > #21 0xc0578e1d in sys_exit (td=Could not find the frame base for "sys_exit". > ) at /ibm/src/sys/kern/kern_exit.c:98 > #22 0xc077fb90 in syscall (frame=0xeb74bd38) > at /ibm/src/sys/i386/i386/trap.c:1035 > #23 0xc07663e0 in Xint0x80_syscall () at /ibm/src/sys/i386/i386/exception.s:196 > #24 0x00000033 in ?? () > Previous frame inner to this frame (corrupt stack?) > (kgdb)Fatal trap 12: page fault while in kernel mode cpuid = 1; apic id = 01 fault virtual address = 0x0 fault code = supervisor write data, page not present instruction pointer = 0x8:0xffffffff80162c11 stack pointer = 0x10:0xffffffffabe679c0 frame pointer = 0x10:0xffffff00030f6c00 code segment = base 0x0, limit 0xfffff, type 0x1b = DPL 0, pres 1, long 1, def32 0, gran 1 processor eflags = interrupt enabled, resume, IOPL = 0 current process = 2 (g_event) trap number = 12 panic: page fault cpuid = 1 Uptime: 2d12h40m24s (da1:dead_sim0:0:0:0): Synchronize cache failed, status == 0x34, scsi status == 0x0 Physical memory: 2807 MB Dumping 460 MB: 445 429 413 397 381 365 349 333 317 301 285 269 253 237 221 205 189 173 157 141 125 109 93 77 61 45 29 13 Reading symbols from /boot/kernel/umass.ko...Reading symbols from /boot/kernel/umass.ko.symbols...done. done. Loaded symbols for /boot/kernel/umass.ko Reading symbols from /boot/kernel/usb.ko...Reading symbols from /boot/kernel/usb.ko.symbols...done. done. Loaded symbols for /boot/kernel/usb.ko Reading symbols from /boot/kernel/sym.ko...Reading symbols from /boot/kernel/sym.ko.symbols...done. done. Loaded symbols for /boot/kernel/sym.ko Reading symbols from /boot/kernel/iicsmb.ko...Reading symbols from /boot/kernel/iicsmb.ko.symbols...done. done. Loaded symbols for /boot/kernel/iicsmb.ko Reading symbols from /boot/kernel/iicbus.ko...Reading symbols from /boot/kernel/iicbus.ko.symbols...done. done. Loaded symbols for /boot/kernel/iicbus.ko Reading symbols from /boot/kernel/smbus.ko...Reading symbols from /boot/kernel/smbus.ko.symbols...done. done. Loaded symbols for /boot/kernel/smbus.ko Reading symbols from /boot/kernel/smb.ko...Reading symbols from /boot/kernel/smb.ko.symbols...done. done. Loaded symbols for /boot/kernel/smb.ko Reading symbols from /boot/kernel/nfsmb.ko...Reading symbols from /boot/kernel/nfsmb.ko.symbols...done. done. Loaded symbols for /boot/kernel/nfsmb.ko Reading symbols from /boot/kernel/k8temp.ko...Reading symbols from /boot/kernel/k8temp.ko.symbols...done. done. Loaded symbols for /boot/kernel/k8temp.ko Reading symbols from /boot/kernel/if_tap.ko...Reading symbols from /boot/kernel/if_tap.ko.symbols...done. done. Loaded symbols for /boot/kernel/if_tap.ko Reading symbols from /boot/kernel/if_nfe.ko...Reading symbols from /boot/kernel/if_nfe.ko.symbols...done. done. Loaded symbols for /boot/kernel/if_nfe.ko Reading symbols from /boot/kernel/geom_mirror.ko...Reading symbols from /boot/kernel/geom_mirror.ko.symbols...done. done. Loaded symbols for /boot/kernel/geom_mirror.ko Reading symbols from /boot/kernel/geom_raid5.ko...Reading symbols from /boot/kernel/geom_raid5.ko.symbols...done. done. Loaded symbols for /boot/kernel/geom_raid5.ko Reading symbols from /boot/kernel/geom_eli.ko...Reading symbols from /boot/kernel/geom_eli.ko.symbols...done. done. Loaded symbols for /boot/kernel/geom_eli.ko Reading symbols from /boot/kernel/crypto.ko...Reading symbols from /boot/kernel/crypto.ko.symbols...done. done. Loaded symbols for /boot/kernel/crypto.ko Reading symbols from /boot/kernel/zlib.ko...Reading symbols from /boot/kernel/zlib.ko.symbols...done. done. Loaded symbols for /boot/kernel/zlib.ko #0 doadump () at pcpu.h:194 194 pcpu.h: No such file or directory. in pcpu.h (kgdb) where #0 doadump () at pcpu.h:194 #1 0x0000000000000004 in ?? () #2 0xffffffff80289290 in boot (howto=260) at /raid1/bsd/src7/sys/kern/kern_shutdown.c:418 #3 0xffffffff802896ad in panic (fmt=0x104 <Address 0x104 out of bounds>) at /raid1/bsd/src7/sys/kern/kern_shutdown.c:572 #4 0xffffffff80494154 in trap_fatal (frame=0xffffff00010e4000, eva=18446742974215571664) at /raid1/bsd/src7/sys/amd64/amd64/trap.c:724 #5 0xffffffff80494525 in trap_pfault (frame=0xffffffffabe67910, usermode=0) at /raid1/bsd/src7/sys/amd64/amd64/trap.c:641 #6 0xffffffff80494ecb in trap (frame=0xffffffffabe67910) at /raid1/bsd/src7/sys/amd64/amd64/trap.c:410 #7 0xffffffff8047ab1e in calltrap () at /raid1/bsd/src7/sys/amd64/amd64/exception.S:169 #8 0xffffffff80162c11 in xpt_done (done_ccb=0xffffff00030f6c00) at /raid1/bsd/src7/sys/cam/cam_xpt.c:4856 #9 0xffffffff8015f775 in cam_periph_runccb (ccb=0xffffff00030f6c00, error_routine=0, camflags=CAM_FLAG_NONE, sense_flags=1, ds=0xffffff00031765a0) at /raid1/bsd/src7/sys/cam/cam_periph.c:878 #10 0xffffffff80175e7f in daclose (dp=Variable "dp" is not available. ) at /raid1/bsd/src7/sys/cam/scsi/scsi_da.c:714 #11 0xffffffff8023a182 in g_disk_access (pp=0xffffff0003c64600, r=Variable "r" is not available. ) at /raid1/bsd/src7/sys/geom/geom_disk.c:152 #12 0xffffffff8023ed53 in g_access (cp=0xffffff0003fb5a00, dcr=-1, dcw=-1, dce=-2) at /raid1/bsd/src7/sys/geom/geom_subr.c:748 #13 0xffffffff8023ed53 in g_access (cp=0xffffff004c625680, dcr=-1, dcw=-1, dce=-1) at /raid1/bsd/src7/sys/geom/geom_subr.c:748 #14 0xffffffff8023f297 in g_wither_geom_close (gp=0xffffff0003e16000, error=6) at /raid1/bsd/src7/sys/geom/geom_subr.c:351 #15 0xffffffffaf73f1fb in g_eli_destroy () from /boot/kernel/geom_eli.ko #16 0xffffffff8023b47e in g_run_events () at /raid1/bsd/src7/sys/geom/geom_event.c:163 #17 0xffffffff8023c8f5 in g_event_procbody () at /raid1/bsd/src7/sys/geom/geom_kern.c:141 #18 0xffffffff802686f2 in fork_exit ( callout=0xffffffff8023c8a0 <g_event_procbody>, arg=0x0, frame=0xffffffffabe67c80) at /raid1/bsd/src7/sys/kern/kern_fork.c:783 #19 0xffffffff8047aeee in fork_trampoline () at /raid1/bsd/src7/sys/amd64/amd64/exception.S:415 #20 0x0000000000000000 in ?? () #21 0x0000000000000000 in ?? () #22 0x0000000000000001 in ?? () #23 0x0000000000000000 in ?? () #24 0x0000000000000000 in ?? () #25 0x0000000000000000 in ?? () #26 0x0000000000000000 in ?? () #27 0x0000000000000000 in ?? () #28 0x0000000000000000 in ?? () #29 0x0000000000000000 in ?? () #30 0x0000000000000000 in ?? () #31 0x0000000000000000 in ?? () #32 0x0000000000000000 in ?? () #33 0x0000000000000000 in ?? () #34 0x0000000000000000 in ?? () #35 0x0000000000000000 in ?? () #36 0x0000000000000000 in ?? () #37 0x0000000000000000 in ?? () #38 0x0000000000000000 in ?? () #39 0x0000000000000000 in ?? () #40 0x0000000000000000 in ?? () #41 0x0000000000000000 in ?? () #42 0x0000000000000000 in ?? () #43 0x0000000000000000 in ?? () #44 0x000000000092b000 in ?? () #45 0xffffffff806b6c80 in tdg_maxid () #46 0xffffffff806c3480 in tdq_cpu () #47 0xffffffff806c3480 in tdq_cpu () #48 0xffffff00010e4000 in ?? () #49 0x000000000000000b in ?? () #50 0xffffffffabe671a8 in ?? () #51 0x0000000000000000 in ?? () #52 0xffffffff802a9854 in sched_switch (td=0xffffffff8023c8a0, newtd=0x0, flags=0) at /raid1/bsd/src7/sys/kern/sched_ule.c:1897