dtrace discuss - Oct 2009 - Is it possible to use DTrace against core file? - Firefox bug

Hi all,

maybe I get this bug
http://crash-stats.mozilla.com/report/index/ebd1ee2b-020a-410e-aef9-6bd942090703?p=1

I want to try investigate, but mdb is cryptic for me after end of part with mdb
core :-)

Here are some outputs :

$ pstack core.firefox/1
core ''core.firefox/1'' of 845:	/usr/lib/firefox/firefox-bin
-----------------  lwp# 1 / thread# 1  --------------------
 feec39f5 _lwp_kill (1, b, 80463d8, fee6affe) + 15
 fee6b00a raise    (b, 80463f0, 0, fd338dea) + 22
 fd338ecd __1cNnsProfileLockSFatalSignalHandler6Fi_v_ (b, 0, 80464e4, fef51000,
80464a0, feeb15ef) + f1
 feebe825 __sighndlr (b, 0, 80464e4, fd338ddc) + 15
 feeb15ef call_user_handler (b) + 2af
 feeb181f sigacthandler (b, 0, 80464e4) + df
 --- called from signal handler with signal 11 (SIGSEGV) ---
 fcb6f639 PR_EnumerateAddrInfo (eda63e00, eda81ba0, 0, 8046754) + 39
 fd4220cb __1cLnsDNSRecordLGetNextAddr6MHpnJPRNetAddr__I_ (eda811e0, 0, 8046754,
fd4221fb) + 77
 fd422219 __1cLnsDNSRecordTGetNextAddrAsString6MrnTnsACString_internal__I_
(eda811e0, de4f00c0, 1, fe6932a8, eda811e0, 5) + 2d
 fe427f9f NS_InvokeByIndex_P (eda811e0, 5, 1, 80469a0) + 51
 fd37b0ea __1cQXPCWrappedNativeKCallMethod6FrnOXPCCallContext_n0AICallMode__i_
(8046b20, 0, db9ff674, 0) + 1406
 fd38432c __1cRXPC_WN_CallMethod6FpnJJSContext_pnIJSObject_Ipi4_i_ (fb663000,
f586f1e0, 0, df8aa8f4, 8046c60, 0) + c4
 fc8deca6 js_Invoke (fb663000, 0, df8aa8ec, 0) + 806
 fc8d6007 js_Interpret (fb663000, fca2b03c, 8046e18, fc8de4b1) + 612f
 fc8ded2a js_Invoke (fb663000, 3, dda1d120, 0) + 88a
 fd37522d
__1cTnsXPCWrappedJSClassKCallMethod6MpnOnsXPCWrappedJS_HpknTXPTMethodDescriptor_pnRnsXPTCMiniVariant__I_
(de8bc730, deeab900, 3, f6528bb8, 80471e0, fef51000) + 1be5
 fd36d5ee
__1cOnsXPCWrappedJSKCallMethod6MHpknTXPTMethodDescriptor_pnRnsXPTCMiniVariant__I_
(deeab900, 3, f6528bb8, 80471e0) + 36
 fe427ee3 __1cSPrepareAndDispatch6FpnOnsXPTCStubBase_IpI_I_ (de658c40, 3,
80472a0) + 18f
 fe427fcd __1cOnsXPTCStubBaseFStub36M_I_ (de658c40, de8bc97c, eda811e0, 0,
fb6781d4, fe6932a8) + 22
 fe427f9f NS_InvokeByIndex_P (de658c40, 3, 3, de8bc9a0) + 51
 fe41855f __1cVnsProxyObjectCallInfoDRun6M_I_ (de8bc9d0, 1, 804733c, 0) + 33
 fe411306 __1cInsThreadQProcessNextEvent6Mipi_I_ (fb8f7600, 1, 804737c,
fe3afe11) + 142
 fe3afe48 __1cVNS_ProcessNextEvent_P6FpnJnsIThread_i_i_ (fb8f7600, 1, 80473b8,
fe244c82) + 44
 fe244ca2 __1cOnsBaseAppShellDRun6M_I_ (f69278d0, fdfbbf4c, 0, fdfbdae4) + 3a
 fdfbdb0c __1cMnsAppStartupDRun6M_I_ (fb8dad60, fb8350c8, fb8ead98, 0) + 34
 fd32f696 XRE_main (1, 8047868, fb8090c0, 2) + 324a
 080517ab main     (1, 8047868, 8047870, 805143d) + 223
 0805147d _start   (1, 804799c, 0, 80479b9, 80479cd, 80479f2) + 7d

$ pflags core.firefox 
core ''core.firefox'' of 845:	/usr/lib/firefox/firefox-bin
	data model = _ILP32  flags = MSACCT|MSFORK
 /1:	flags = 0
	sigmask = 0xfffffeff,0x0000ffff  cursig = SIGSEGV
 /2:	flags = STOPPED  lwp_park(0x0,0x0,0x0)
	why = PR_SUSPENDED
 /3:	flags = DETACH|STOPPED  lwp_park(0x0,0xf67ffe58,0x0)
	why = PR_SUSPENDED
 /4:	flags = STOPPED  lwp_park(0x0,0xf5ffed78,0x0)
	why = PR_SUSPENDED
 /10:	flags = STOPPED  lwp_park(0x0,0x0,0x0)
	why = PR_SUSPENDED
 /11:	flags = STOPPED  lwp_park(0x0,0x0,0x0)
	why = PR_SUSPENDED
 /12:	flags = STOPPED  lwp_park(0x0,0x0,0x0)
	why = PR_SUSPENDED
 /13:	flags = STOPPED  lwp_park(0x0,0x0,0x0)
	why = PR_SUSPENDED
 /116:	flags = DETACH|STOPPED  lwp_park(0x0,0x0,0x0)
	why = PR_SUSPENDED
 /117:	flags = DETACH|STOPPED  lwp_park(0x0,0x0,0x0)
	why = PR_SUSPENDED
 /597:	flags = DETACH|STOPPED  lwp_park(0x0,0xf2a0be38,0x0)
	why = PR_SUSPENDED
 /601:	flags = STOPPED  lwp_park(0x0,0xf290cd48,0x0)
	why = PR_SUSPENDED


$ mdb core.firefox 
mdb: core file data for mapping at d9200000 not saved: Bad address
mdb: core file data for mapping at d9268000 not saved: Bad address
mdb: core file data for mapping at d9304000 not saved: Bad address
mdb: core file data for mapping at d936b000 not saved: Bad address
mdb: core file data for mapping at d93fa000 not saved: Bad address
mdb: core file data for mapping at e261b000 not saved: Bad address
mdb: core file data for mapping at e2643000 not saved: Bad address
mdb: core file data for mapping at e26df000 not saved: Bad address
mdb: core file data for mapping at e273d000 not saved: Bad address
mdb: core file data for mapping at e281b000 not saved: Bad address
mdb: core file data for mapping at e293f000 not saved: Bad address
mdb: core file data for mapping at e2991000 not saved: Bad address
mdb: core file data for mapping at e29c7000 not saved: Bad address
mdb: core file data for mapping at e2e80000 not saved: Bad address
mdb: core file data for mapping at e339e000 not saved: Bad address
Loading modules: [ libuutil.so.1 ld.so.1 ]
> ::stack
libc_hwcap1.so.1`_lwp_kill+0x15(1, b, 80463d8, fee6affe)
libc_hwcap1.so.1`raise+0x22(b, 80463f0, 0, fd338dea)
libxul.so`__1cNnsProfileLockSFatalSignalHandler6Fi_v_+0xf1(b, 0, 80464e4,
fef51000, 80464a0, feeb15ef)
libc_hwcap1.so.1`__sighndlr+0x15(b, 0, 80464e4, fd338ddc)
libc_hwcap1.so.1`call_user_handler+0x2af(b)
libc_hwcap1.so.1`sigacthandler+0xdf(b, 0, 80464e4)
libnspr4.so`PR_EnumerateAddrInfo+0x39(eda63e00, eda81ba0, 0, 8046754)
libxul.so`__1cLnsDNSRecordLGetNextAddr6MHpnJPRNetAddr__I_+0x77(eda811e0, 0,
8046754, fd4221fb)
libxul.so`__1cLnsDNSRecordTGetNextAddrAsString6MrnTnsACString_internal__I_+0x2d(eda811e0,
de4f00c0, 1, fe6932a8, eda811e0, 5)
libxul.so`NS_InvokeByIndex_P+0x51(eda811e0, 5, 1, 80469a0)
libxul.so`__1cQXPCWrappedNativeKCallMethod6FrnOXPCCallContext_n0AICallMode__i_+0x1406(8046b20,
0, db9ff674, 0)
libxul.so`__1cRXPC_WN_CallMethod6FpnJJSContext_pnIJSObject_Ipi4_i_+0xc4(fb663000,
f586f1e0, 0, df8aa8f4, 8046c60, 0)
libmozjs.so`js_Invoke+0x806(fb663000, 0, df8aa8ec, 0)
libmozjs.so`js_Interpret+0x612f(fb663000, fca2b03c, 8046e18, fc8de4b1)
libmozjs.so`js_Invoke+0x88a(fb663000, 3, dda1d120, 0)
libxul.so`__1cTnsXPCWrappedJSClassKCallMethod6MpnOnsXPCWrappedJS_HpknTXPTMethodDescriptor_pnRnsXPTCMiniVariant__I_+0x1be5(de8bc730,
deeab900, 3, f6528bb8, 80471e0, fef51000)
libxul.so`__1cOnsXPCWrappedJSKCallMethod6MHpknTXPTMethodDescriptor_pnRnsXPTCMiniVariant__I_+0x36(deeab900,
3, f6528bb8, 80471e0)
libxul.so`__1cSPrepareAndDispatch6FpnOnsXPTCStubBase_IpI_I_+0x18f(de658c40, 3,
80472a0)
libxul.so`__1cOnsXPTCStubBaseFStub36M_I_+0x22(de658c40, de8bc97c, eda811e0, 0,
fb6781d4, fe6932a8)
libxul.so`NS_InvokeByIndex_P+0x51(de658c40, 3, 3, de8bc9a0)
libxul.so`__1cVnsProxyObjectCallInfoDRun6M_I_+0x33(de8bc9d0, 1, 804733c, 0)
libxul.so`__1cInsThreadQProcessNextEvent6Mipi_I_+0x142(fb8f7600, 1, 804737c,
fe3afe11)
libxul.so`__1cVNS_ProcessNextEvent_P6FpnJnsIThread_i_i_+0x44(fb8f7600, 1,
80473b8, fe244c82)
libxul.so`__1cOnsBaseAppShellDRun6M_I_+0x3a(f69278d0, fdfbbf4c, 0, fdfbdae4)
libxul.so`__1cMnsAppStartupDRun6M_I_+0x34(fb8dad60, fb8350c8, fb8ead98, 0)
libxul.so`XRE_main+0x324a(1, 8047868, fb8090c0, 2)
main+0x223(1, 8047868, 8047870, 805143d)
_start+0x7d(1, 804799c, 0, 80479b9, 80479cd, 80479f2)
> 
So here I''m stuck. If I''m correct then FF crashed on this
https://developer.mozilla.org/En/PR_EnumerateAddrInfo which is in this main
function http://mxr.mozilla.org/firefox/source/toolkit/xre/nsAppRunner.cpp#1831
but I can''t see addresses in stack output of mdb as in this post
http://solaristhings.blogspot.com/2006/08/dont-be-afraid-of-mdb.html  I can see
library names. So what''s next? Is there a way to get more "human
readable" output with DTrace for similar cases?

Thx a lot
-- 
This message posted from opensolaris.org

Tomas Bodzar wrote:
> Hi all,
> Is it possible to use DTrace against core file? -	Firefox bug
>   
No, as it''s not running.
> So here I''m stuck. If I''m correct then FF crashed on this
https://developer.mozilla.org/En/PR_EnumerateAddrInfo which is in this main
function http://mxr.mozilla.org/firefox/source/toolkit/xre/nsAppRunner.cpp#1831
but I can''t see addresses in stack output of mdb as in this post
http://solaristhings.blogspot.com/2006/08/dont-be-afraid-of-mdb.html  I can see
library names. So what''s next? Is there a way to get more "human
readable" output with DTrace for similar cases?
>   
Run the output through c++filt to demange the C++ names.

You''re probably going to be better off using a debugger which is part
of
the compiler toolchain used to build it, such as dbx or gdb as appropriate.

-- 
Andrew

I used "workaround" in one terminal I used pstack on core file to get
addresses and in second terminal I used mdb for examine core file. But assemble
language is secret for me so I can only say that this looks like bug which I
mentioned above and problem is in library libnspr4.so and function
https://developer.mozilla.org/En/PR_EnumerateAddrInfo

or maybe I''m totally out in this issue :-)

I tried dbx from Sun Studio 12 but not much better then mdb. Still this assemble
stuff :-)
-- 
This message posted from opensolaris.org