Hi,
when using the ustack action on the latest FreeBSD8.2 i386 the kernel
panics.
Here is the information I could gather:
let me know if I can provide more information. (i.e., i have the full crash
information 80+mbs handy)
Here is the backtrace:
Fatal trap 12: page fault while in kernel mode
cpuid = 0; apic id = 00
fault virtual address = 0x108
fault code = supervisor write, page not present
instruction pointer = 0x20:0xc11012d7
stack pointer = 0x28:0xcd3ed9f4
frame pointer = 0x28:0xcd3eda0c
code segment = base 0x0, limit 0xfffff, type 0x1b
= DPL 0, pres 1, def32 1, gran 1
processor eflags = resume, IOPL = 0
current process = 1132 (nc)
trap number = 12
panic: page fault
cpuid = 0
KDB: stack backtrace:
#0 0xc09036a7 at kdb_backtrace+0x47
#1 0xc08d1a07 at panic+0x117
#2 0xc0c158c3 at trap_fatal+0x323
#3 0xc0c15bc0 at trap_pfault+0x2f0
#4 0xc0c1612a at trap+0x48a
#5 0xc0bfc97c at calltrap+0x6
#6 0xc10e992b at dtrace_panic+0x1b
#7 0xc10e995d at dtrace_assfail+0x2d
#8 0xc10fb28c at dtrace_probe+0x135c
#9 0xc1237f72 at systrace_probe+0x62
#10 0xc090f63f at syscallenter+0x47f
#11 0xc0c15c14 at syscall+0x34
#12 0xc0bfca11 at Xint0x80_syscall+0x21
Uptime: 3m0s
Physical memory: 239 MB
Dumping 79 MB: 64 48 32 16
Steps To reproduce:
the dtrace program (i.e., test.d) was:
#!/usr/sbin/dtrace -s
syscall::accept:return
/ execname == "nc" /
{
printf("%s accept:return\n", execname);
ustack();
}
% dtrace -s test.d
then running
% nc -vl 8080
on one shell and
% nc localhost 8080
on another would make the kernel panic
thanks
--m
-------------- next part --------------
A non-text attachment was scrubbed...
Name: core.txt.0
Type: application/octet-stream
Size: 65864 bytes
Desc: not available
Url :
http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20110726/997a7721/core.txt-0001.obj
-------------- next part --------------
A non-text attachment was scrubbed...
Name: info.0
Type: application/octet-stream
Size: 423 bytes
Desc: not available
Url :
http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20110726/997a7721/info-0001.obj
on 26/07/2011 04:20 maestro something said the following:> Hi, > > when using the ustack action on the latest FreeBSD8.2 i386 the kernel > panics. > > Here is the information I could gather: > > let me know if I can provide more information. (i.e., i have the full crash > information 80+mbs handy)Use kgdb on the crash dump, go to the dtrace_probe frame and see which exactly assert fails and why.> Here is the backtrace: > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address = 0x108 > fault code = supervisor write, page not present > instruction pointer = 0x20:0xc11012d7 > stack pointer = 0x28:0xcd3ed9f4 > frame pointer = 0x28:0xcd3eda0c > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = resume, IOPL = 0 > current process = 1132 (nc) > trap number = 12 > panic: page fault > cpuid = 0 > KDB: stack backtrace: > #0 0xc09036a7 at kdb_backtrace+0x47 > #1 0xc08d1a07 at panic+0x117 > #2 0xc0c158c3 at trap_fatal+0x323 > #3 0xc0c15bc0 at trap_pfault+0x2f0 > #4 0xc0c1612a at trap+0x48a > #5 0xc0bfc97c at calltrap+0x6 > #6 0xc10e992b at dtrace_panic+0x1b > #7 0xc10e995d at dtrace_assfail+0x2d > #8 0xc10fb28c at dtrace_probe+0x135c > #9 0xc1237f72 at systrace_probe+0x62 > #10 0xc090f63f at syscallenter+0x47f > #11 0xc0c15c14 at syscall+0x34 > #12 0xc0bfca11 at Xint0x80_syscall+0x21 > Uptime: 3m0s > Physical memory: 239 MB > Dumping 79 MB: 64 48 32 16 > > > Steps To reproduce: > > the dtrace program (i.e., test.d) was: > > #!/usr/sbin/dtrace -s > > syscall::accept:return > / execname == "nc" / > { > printf("%s accept:return\n", execname); > ustack(); > } > > % dtrace -s test.d > > then running > % nc -vl 8080 > on one shell and > % nc localhost 8080 > on another would make the kernel panic-- Andriy Gapon
on 30/07/2011 00:27 maestro something said the following:> Hi, > > trying to do so I don't really find my way around. This is what I get when I run > kgdb > > On startup the assert frame is #7 and the probe frame is #8. >[snip]> kernel trap 12 with interrupts disabled > > > Fatal trap 12: page fault while in kernel mode > cpuid = 0; apic id = 00 > fault virtual address = 0x108 > fault code = supervisor write, page not present > instruction pointer = 0x20:0xc11012d7 > stack pointer = 0x28:0xcd3ed9f4 > frame pointer = 0x28:0xcd3eda0c > code segment = base 0x0, limit 0xfffff, type 0x1b > = DPL 0, pres 1, def32 1, gran 1 > processor eflags = resume, IOPL = 0 > current process = 1132 (nc) > trap number = 12 > panic: page fault > cpuid = 0 > KDB: stack backtrace: > #0 0xc09036a7 at kdb_backtrace+0x47 > #1 0xc08d1a07 at panic+0x117 > #2 0xc0c158c3 at trap_fatal+0x323 > #3 0xc0c15bc0 at trap_pfault+0x2f0 > #4 0xc0c1612a at trap+0x48a > #5 0xc0bfc97c at calltrap+0x6 > #6 0xc10e992b at dtrace_panic+0x1b > #7 0xc10e995d at dtrace_assfail+0x2d > #8 0xc10fb28c at dtrace_probe+0x135c > #9 0xc1237f72 at systrace_probe+0x62 > #10 0xc090f63f at syscallenter+0x47f > #11 0xc0c15c14 at syscall+0x34 > #12 0xc0bfca11 at Xint0x80_syscall+0x21 > Uptime: 3m0s > Physical memory: 239 MB > Dumping 79 MB: 64 48 32 16[snip]> #0 doadump () at pcpu.h:231 > 231 __asm("movl %%fs:0,%0" : "=r" (td)); > > once I'm in the kgdb console i type where and all of a sudden the stack trace > has only 7 frames... that does not seem correct. Furthermore, the "Previous > frame inner to this frame (corrupt stack?)" error message is not too encuraging > either. > > (kgdb) where > #0 doadump () at pcpu.h:231 > #1 0xc08d17a3 in boot (howto=260) at /usr/src/sys/kern/kern_shutdown.c:419 > #2 0xc08d1a40 in panic (fmt=Variable "fmt" is not available. > ) at /usr/src/sys/kern/kern_shutdown.c:592 > #3 0xc0c158c3 in trap_fatal (frame=0xcd3ed9b4, eva=264) at > /usr/src/sys/i386/i386/trap.c:946 > #4 0xc0c15bc0 in trap_pfault (frame=0xcd3ed9b4, usermode=0, eva=264) at > /usr/src/sys/i386/i386/trap.c:859 > #5 0xc0c1612a in trap (frame=0xcd3ed9b4) at /usr/src/sys/i386/i386/trap.c:532 > #6 0xc0bfc97c in calltrap () at /usr/src/sys/i386/i386/exception.s:166 > #7 0xc11012d7 in dtrace_panic_trigger () from /boot/kernel/dtrace.ko > Previous frame inner to this frame (corrupt stack?) > (kgdb) > > what am I doing wrong and what do I have to do to get the specific assert that > fails?I am not quite sure. Apparently there is some issue with either what information we store in a crash dump and how, or with how kgdb interprets that information, or with a combination of both... Perhaps this is a result of -O2 optimization and -O1 would improve the situation - not sure again... Meanwhile, there is something simple that you can do without much hassle: (kgdb) list *dtrace_probe+0x135c where the address is taken from the backtrace printed by panic(9).> On Tue, Jul 26, 2011 at 4:05 AM, Andriy Gapon <avg@freebsd.org > <mailto:avg@freebsd.org>> wrote: > > on 26/07/2011 04:20 maestro something said the following: > > Hi, > > > > when using the ustack action on the latest FreeBSD8.2 i386 the kernel > > panics. > > > > Here is the information I could gather: > > > > let me know if I can provide more information. (i.e., i have the full crash > > information 80+mbs handy) > > Use kgdb on the crash dump, go to the dtrace_probe frame and see which exactly > assert fails and why. > > > Here is the backtrace: > > > > Fatal trap 12: page fault while in kernel mode > > cpuid = 0; apic id = 00 > > fault virtual address = 0x108 > > fault code = supervisor write, page not present > > instruction pointer = 0x20:0xc11012d7 > > stack pointer = 0x28:0xcd3ed9f4 > > frame pointer = 0x28:0xcd3eda0c > > code segment = base 0x0, limit 0xfffff, type 0x1b > > = DPL 0, pres 1, def32 1, gran 1 > > processor eflags = resume, IOPL = 0 > > current process = 1132 (nc) > > trap number = 12 > > panic: page fault > > cpuid = 0 > > KDB: stack backtrace: > > #0 0xc09036a7 at kdb_backtrace+0x47 > > #1 0xc08d1a07 at panic+0x117 > > #2 0xc0c158c3 at trap_fatal+0x323 > > #3 0xc0c15bc0 at trap_pfault+0x2f0 > > #4 0xc0c1612a at trap+0x48a > > #5 0xc0bfc97c at calltrap+0x6 > > #6 0xc10e992b at dtrace_panic+0x1b > > #7 0xc10e995d at dtrace_assfail+0x2d > > #8 0xc10fb28c at dtrace_probe+0x135c > > #9 0xc1237f72 at systrace_probe+0x62 > > #10 0xc090f63f at syscallenter+0x47f > > #11 0xc0c15c14 at syscall+0x34 > > #12 0xc0bfca11 at Xint0x80_syscall+0x21 > > Uptime: 3m0s > > Physical memory: 239 MB > > Dumping 79 MB: 64 48 32 16 > > > > > > Steps To reproduce: > > > > the dtrace program (i.e., test.d) was: > > > > #!/usr/sbin/dtrace -s > > > > syscall::accept:return > > / execname == "nc" / > > { > > printf("%s accept:return\n", execname); > > ustack(); > > } > > > > % dtrace -s test.d > > > > then running > > % nc -vl 8080 > > on one shell and > > % nc localhost 8080 > > on another would make the kernel panic-- Andriy Gapon
Hi, on 30/07/2011 23:03 maestro something said the following:> > (kgdb) list *dtrace_probe+0xfd6 > > No source file for address 0xc10fa6a6. > > Are the sources on the same machine? > This is probably the last idea from me. >Yes all is done on the same (virtual 32bit) machine. Sources are there too. If i do make installkernel KERNCONF=MYKERNEL the kernel should be installed right? (I assume so b/c /boot/kernel/kernel has the correct new timestamp) cheers -m
Hi, I just tried your test on -STABLE + 1 fix from current [1] and got no trap. dtrace: buffer size lowered to 2m CPU ID FUNCTION:NAME 4 42224 accept:return nc accept:return Assertion failed: (dpr != NULL), file /usr/src/cddl/lib/libdtrace/../../../cddl/contrib/opensolaris/lib/libdtrace/common/dt_proc.c, line 751. Abort (core dumped) Also 1st dtrace was killed as well. I tried 2 times with the same result. Box is amd64. This is bt from gdb: (gdb) bt #0 0x0000000800fa2f9c in thr_kill () from /lib/libc.so.7 #1 0x000000080103f6cb in abort () from /lib/libc.so.7 #2 0x0000000800786758 in dt_proc_lookup () from /lib/libdtrace.so.2 #3 0x00000008007867db in dt_proc_lock () from /lib/libdtrace.so.2 #4 0x00000008007a5bcb in dt_print_ustack () from /lib/libdtrace.so.2 #5 0x00000008007a8504 in dt_print_agg () from /lib/libdtrace.so.2 #6 0x00000008007a88ca in dtrace_consume () from /lib/libdtrace.so.2 #7 0x000000080077e1a2 in dtrace_work () from /lib/libdtrace.so.2 #8 0x00000000004044ba in ?? () #9 0x000000000040222e in ?? () [skip] [1] http://www.freebsd.org/cgi/query-pr.cgi?pr=kern/159064 On 07/26/2011 03:20 AM, maestro something wrote:> Hi, > > when using the ustack action on the latest FreeBSD8.2 i386 the kernel > panics.
on 03/08/2011 01:46 maestro something said the following:> What do I have to recompile for the patch to be picked up? > make libproc in /usr/src/libCorrect way is: $ cd /usr/src/lib/libproc $ make obj $ make depend $ make $ make install -- Andriy Gapon