Hi, I am working with a customer who is trying to capture synch data for his application and plockstat coredumps with the following stack: It looks like a NULL string is passed to strlen, thus causing a coredump. Is this a known bug? thanks in advance for your help. Geetha t at 1 (l at 1) terminated by signal SEGV (no mapping at the fault address) 0xffffffff7ed39910: strlen+0x0050: ld [%o2], %o1 (dbx) where -h current thread: t at 1 =>[1] strlen(0x0, 0x53, 0x0, 0x0, 0x0, 0x53), at 0xffffffff7ed39910 [2] _ndoprnt(0x100003b7a, 0xffffffff7fffeca8, 0xffffffff7eda06fc, 0xffffffff7fffe359, 0x0, 0x100003b79), at 0xffffffff7eda246c [3] snprintf(0xffffffff7fffefc0, 0x0, 0x100003b78, 0x0, 0xffffffff7fffed00, 0x100003000), at 0xffffffff7eda4804 [4] 0x100001c8c(0x1003bd170, 0x10075c8b0, 0xffffffff7fffefc0, 0x28, 0x1, 0x1d), at 0x100001c8b [5] 0x100001f50(0x1003ec008, 0x3, 0x1003ec010, 0x1, 0x1003ec320, 0x40), at 0x100001f4f [6] dt_aggregate_walk_sorted(0x1001094d0, 0x100001d98, 0x0, 0xffffffff7f2242e4, 0x7, 0x10040fd10), at 0xffffffff7f224a40 [7] main(0x1001051f8, 0x1001051f8, 0x100105000, 0x1, 0x100105000, 0x100000), at 0x100002ac4 #### (dbx) dis 0xffffffff7eda246c 0xffffffff7eda246c: _ndoprnt+0x1c04: call _PROCEDURE_LINKAGE_TABLE_+0x640 [PLT] ! 0xffffffff7eee6a40 0xffffffff7eda2470: _ndoprnt+0x1c08: mov %i4, %o0 ### %i4 is NULL when I dump the registers from the frame.
Hi Geetha, Not a known bug; can you send me the core file? Adam On Wed, Oct 26, 2005 at 03:28:54PM -0700, Geetha Vallabhaneni wrote:> Hi, > > I am working with a customer who is trying to capture synch data for his application and plockstat coredumps with the following stack: > > It looks like a NULL string is passed to strlen, thus causing a coredump. Is this a known bug? > > thanks in advance for your help. > Geetha > > t at 1 (l at 1) terminated by signal SEGV (no mapping at the fault address) > 0xffffffff7ed39910: strlen+0x0050: ld [%o2], %o1 > (dbx) where -h > current thread: t at 1 > =>[1] strlen(0x0, 0x53, 0x0, 0x0, 0x0, 0x53), at 0xffffffff7ed39910 > [2] _ndoprnt(0x100003b7a, 0xffffffff7fffeca8, 0xffffffff7eda06fc, 0xffffffff7fffe359, 0x0, 0x100003b79), at 0xffffffff7eda246c > [3] snprintf(0xffffffff7fffefc0, 0x0, 0x100003b78, 0x0, 0xffffffff7fffed00, 0x100003000), at 0xffffffff7eda4804 > [4] 0x100001c8c(0x1003bd170, 0x10075c8b0, 0xffffffff7fffefc0, 0x28, 0x1, 0x1d), at 0x100001c8b > [5] 0x100001f50(0x1003ec008, 0x3, 0x1003ec010, 0x1, 0x1003ec320, 0x40), at 0x100001f4f > [6] dt_aggregate_walk_sorted(0x1001094d0, 0x100001d98, 0x0, 0xffffffff7f2242e4, 0x7, 0x10040fd10), at 0xffffffff7f224a40 > [7] main(0x1001051f8, 0x1001051f8, 0x100105000, 0x1, 0x100105000, 0x100000), at 0x100002ac4 > > > #### > (dbx) dis 0xffffffff7eda246c > 0xffffffff7eda246c: _ndoprnt+0x1c04: call _PROCEDURE_LINKAGE_TABLE_+0x640 [PLT] ! 0xffffffff7eee6a40 > 0xffffffff7eda2470: _ndoprnt+0x1c08: mov %i4, %o0 > > ### %i4 is NULL when I dump the registers from the frame. > > > _______________________________________________ > dtrace-discuss mailing list > dtrace-discuss at opensolaris.org-- Adam Leventhal, Solaris Kernel Development http://blogs.sun.com/ahl
Was this fixed - I get a different symptom but also a coredump when I run plockstat as below (I am giving the context for you using other system commands). May be this is not the same bug. BTW, I am not using OpenSolaris - all Nevada bits. [bash:500] whoami root [bash:501] uname -a SunOS serverxx.vonagenetworks.net 5.11 snv_38 sun4v sparc SUNW,Sun-Fire-T1000 **** [bash:502] plockstat -v -e 4 -s 5 -A -p `pgrep myprogram` **** plockstat: tracing enabled for pid 9060 **** plockstat: processing aborted: Abort due to drop [bash:503] pgrep myprogram 9060 [bash:504] This message posted from opensolaris.org
If you''re get a data drop, you can increase the size of the aggregation buffer using the -x aggsize=<size> option (which is the same for dtrace(1M)). The default is 256k. Adam On Tue, Sep 12, 2006 at 09:26:59AM -0700, Pankaj Shroff wrote:> Was this fixed - I get a different symptom but also a coredump when I run plockstat as below (I am giving the context for you using other system commands). May be this is not the same bug. BTW, I am not using OpenSolaris - all Nevada bits. > > [bash:500] whoami > root > [bash:501] uname -a > SunOS serverxx.vonagenetworks.net 5.11 snv_38 sun4v sparc SUNW,Sun-Fire-T1000 > **** [bash:502] plockstat -v -e 4 -s 5 -A -p `pgrep myprogram` > **** plockstat: tracing enabled for pid 9060 > **** plockstat: processing aborted: Abort due to drop > [bash:503] pgrep myprogram > 9060 > [bash:504] > > > This message posted from opensolaris.org > _______________________________________________ > dtrace-discuss mailing list > dtrace-discuss at opensolaris.org-- Adam Leventhal, Solaris Kernel Development http://blogs.sun.com/ahl
Hi Pankaj, You''re hitting this but for which Robert Harris sent me a code review this morning: 6296612 plockstat dumps core on second call Adam P.S. to anyone who gets a core file or crash dump, we do want them, but please don''t send them out to the whole list since some people get cranky about big attachments. On Tue, Sep 12, 2006 at 01:47:32PM -0400, Pankaj Shroff wrote:> Adam, > > Thanks, that did get me further - but I still get a core dump with a Segmentation Fault. Here is the output. The core file for last run in the below output is attached. I ran it three times once with -A, then with -H and -C. > > It seems -C consistently dumps core immediately. -H dumps core in the middle - and since -A simply calls mutex-hold before mutex-acquire - it behaves identical to -H. > > > [media-relay-02.klga1:534] plockstat -e 1 -A -v -x aggsize=10m -x bufsize=100m -p `pgrep mediarelay` > plockstat: tracing enabled for pid 1356 > 0 > Mutex hold > > Count nsec Lock Caller > ------------------------------------------------------------------------------- > 111 426000 mediarelay`LogLock mediarelay`__1cDlog6FiipcE_v_+0x174 > 285 86615 mediarelay`PortDataLock+0x29328 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 285 83406 mediarelay`PortDataLock+0x329e8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 285 82546 mediarelay`PortDataLock+0x4cc8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 285 80608 mediarelay`PortDataLock+0x2820 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 281 81616 mediarelay`PortDataLock+0x5508 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 285 80212 mediarelay`PortDataLock+0x2e098 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 285 80004 mediarelay`PortDataLock+0x33a68 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 300 74673 mediarelay`UnauthIPHash+0x8 mediarelay`__1cLGetFromHash6FpnGHash_t_pv_i_+0x80 > 113 196502 libc.so.1`_xftab+0xb0 mediarelay`__1cDlog6FiipcE_v_+0xe8 > 285 77012 mediarelay`PortDataLock+0x325f8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 285 76807 mediarelay`PortDataLock+0x33af8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 94 231041 mediarelay`PortDataLock+0x9cf0 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 285 75637 mediarelay`PortDataLock+0x5550 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > Segmentation Fault (core dumped) > [media-relay-02.klga1:535] plockstat -e 1 -A -v -x aggsize=100m -x bufsize=100m -p `pgrep mediarelay` > plockstat: tracing enabled for pid 1356 > 0 > Mutex hold > > Count nsec Lock Caller > ------------------------------------------------------------------------------- > 96 280927 mediarelay`PortDataLock+0x9348 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 287 91347 mediarelay`PortDataLock+0x325f8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 287 86818 mediarelay`PortDataLock+0x5bf8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 287 86000 mediarelay`PortDataLock+0x29328 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 287 83733 mediarelay`PortDataLock+0x34848 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 92 249987 mediarelay`LogLock mediarelay`__1cDlog6FiipcE_v_+0x174 > 287 78108 mediarelay`PortDataLock+0x5da8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 287 78009 mediarelay`PortDataLock+0x34ba8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 300 73320 mediarelay`UnauthIPHash+0x8 mediarelay`__1cLGetFromHash6FpnGHash_t_pv_i_+0x80 > 286 73075 mediarelay`PortDataLock+0x2e098 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 287 72047 mediarelay`PortDataLock+0x2820 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 235 84752 mediarelay`PortDataLock+0x5ac0 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 92 216424 mediarelay`PortDataLock+0x2100 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 235 79105 mediarelay`PortDataLock+0x345d8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 213 86087 mediarelay`PortDataLock+0x326e8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 211 85669 mediarelay`PortDataLock+0x4b48 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 212 83957 mediarelay`PortDataLock+0x73e0 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 213 82722 mediarelay`PortDataLock+0x3ceb8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 213 81356 mediarelay`PortDataLock+0x9f30 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 65 266489 mediarelay`PortDataLock+0x6ba0 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 213 79485 mediarelay`PortDataLock+0x37818 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 92 182855 mediarelay`LogLock mediarelay`__1cDlog6FiipcE_v_+0x174 > 179 93271 mediarelay`PortDataLock+0x34908 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 95 173841 mediarelay`PortDataLock+0x34c8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 179 87869 mediarelay`PortDataLock+0x2e608 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 179 87636 mediarelay`PortDataLock+0x5c58 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 96 161266 mediarelay`PortDataLock+0x5538 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 186 80359 mediarelay`PortDataLock+0x2688 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 186 79890 mediarelay`PortDataLock+0x2dd68 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > Segmentation Fault (core dumped) > [media-relay-02.klga1:536] plockstat -e 1 -A -C -x aggsize=100m -x bufsize=100m -p `pgrep mediarelay` > ^C > [media-relay-02.klga1:537] plockstat -e 1 -C -x aggsize=100m -x bufsize=100m -p `pgrep mediarelay` > 0 > Mutex block > > Count nsec Lock Caller > ------------------------------------------------------------------------------- > Segmentation Fault (core dumped) > [media-relay-02.klga1:538] plockstat -e 1 -H -x aggsize=100m -x bufsize=100m -p `pgrep mediarelay` > 0 > Mutex hold > > Count nsec Lock Caller > ------------------------------------------------------------------------------- > 287 84185 mediarelay`PortDataLock+0x29328 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 286 83732 mediarelay`PortDataLock+0x2e098 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 90 264659 mediarelay`LogLock mediarelay`__1cDlog6FiipcE_v_+0x174 > 15 1531288 mediarelay`CallIDHash+0x8 mediarelay`__1cOInsertCallData6FpnOCallDataStruct__v_+0x258 > 286 75901 mediarelay`PortDataLock+0x34ba8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 287 75573 mediarelay`PortDataLock+0x35ef8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 287 74264 mediarelay`PortDataLock+0x6750 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 301 69628 mediarelay`UnauthIPHash+0x8 mediarelay`__1cLGetFromHash6FpnGHash_t_pv_i_+0x80 > 286 72101 mediarelay`PortDataLock+0x5da8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 246 81573 mediarelay`PortDataLock+0x2820 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 249 77563 mediarelay`PortDataLock+0x6eb8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 248 76414 mediarelay`PortDataLock+0x36dc8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 213 84743 mediarelay`PortDataLock+0x37818 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 212 83796 mediarelay`PortDataLock+0x73e0 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 96 184212 mediarelay`PortDataLock+0x2b638 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 213 79347 mediarelay`PortDataLock+0x3ceb8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 213 74962 mediarelay`PortDataLock+0x9f30 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 185 80177 mediarelay`PortDataLock+0x2688 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 184 79544 mediarelay`PortDataLock+0x2dd68 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 185 78712 mediarelay`PortDataLock+0x6d98 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 179 79487 mediarelay`PortDataLock+0x326e8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 90 157577 mediarelay`LogLock mediarelay`__1cDlog6FiipcE_v_+0x174 > 221 63409 mediarelay`PortDataLock+0x6ee8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 179 78041 mediarelay`PortDataLock+0x4b48 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 96 136540 mediarelay`PortDataLock+0x2adc8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 90 143252 mediarelay`LogLock mediarelay`__1cDlog6FiipcE_v_+0x174 > 96 133822 mediarelay`PortDataLock+0x66c0 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 95 112919 mediarelay`PortDataLock+0x371e8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > 134 79917 mediarelay`PortDataLock+0x373c8 mediarelay`__1cJRTPListen6Fpv_0_+0x708 > Segmentation Fault (core dumped) > > > > Pankaj Shroff > Engineer, Network Architecture > VONAGE | 23 Main Street | Holmdel, NJ 07733 > t: 732.203.7083 | c: 848.219.7291 > e: pankaj.shroff at vonage.com > w: http://www.vonage.com/ > > > NOTE: The information contained in this email message is considered confidential and proprietary to the sender and is intended solely for review and use by the named recipient. Any unauthorized review, use or distribution is strictly prohibited. If you have received this message in error, please advise the sender by reply email and delete the message. > > > > > -----Original Message----- > From: Adam Leventhal [mailto:ahl at eng.sun.com] > Sent: Tue 9/12/2006 12:57 PM > To: Pankaj Shroff > Cc: dtrace-discuss at opensolaris.org > Subject: Re: [dtrace-discuss] Re: plockstat coredump > > If you''re get a data drop, you can increase the size of the aggregation buffer > using the -x aggsize=<size> option (which is the same for dtrace(1M)). The > default is 256k. > > Adam > > On Tue, Sep 12, 2006 at 09:26:59AM -0700, Pankaj Shroff wrote: > > Was this fixed - I get a different symptom but also a coredump when I run plockstat as below (I am giving the context for you using other system commands). May be this is not the same bug. BTW, I am not using OpenSolaris - all Nevada bits. > > > > [bash:500] whoami > > root > > [bash:501] uname -a > > SunOS serverxx.vonagenetworks.net 5.11 snv_38 sun4v sparc SUNW,Sun-Fire-T1000 > > **** [bash:502] plockstat -v -e 4 -s 5 -A -p `pgrep myprogram` > > **** plockstat: tracing enabled for pid 9060 > > **** plockstat: processing aborted: Abort due to drop > > [bash:503] pgrep myprogram > > 9060 > > [bash:504] > > > > > > This message posted from opensolaris.org > > _______________________________________________ > > dtrace-discuss mailing list > > dtrace-discuss at opensolaris.org > > -- > Adam Leventhal, Solaris Kernel Development http://blogs.sun.com/ahl >-- Adam Leventhal, Solaris Kernel Development http://blogs.sun.com/ahl