John Rice
2007-Feb-24 10:42 UTC
[dtrace-discuss] Hotspot provider system crashes, class data sharing and automatic server detection gone wrong
HI,
I''ve been looking at the hotspot provider and wanted to demo some of
it''s features. Running on nevada 56 I''m getting a lot of
system hangs
:( The system was rock solid until I started playing with the hotspot
provider, it seems particularly unhappy about the class-loaded probe for
some reason, though it''s hung with other method-entry probes. Anyone
else seeing this?
The following script was run, as I wanted to demonstrate Class data sharing.
$ dtrace -Zqn ''hotspot*:::class-loaded{ @loaded[arg2, copyinstr(arg0,
arg1), arg3? "Shared":"Unique"] = count();
@total[arg3?"Shared":"Unique"] = count();}
END{printa("%-8d %40s %-10s
%8 at d\n", @loaded); printa("%-20s %@d\n", at total);}''
$ java -jar /usr/java/demo/jfc/SwingSet2/SwingSet2.jar
Strangely all the classes where unique. Checked to see if I had a class
archive, I do:
/usr/jdk/instances/jdk1.6.0/jre/lib/i386/client/classes.jsa
Then looked to see what the hotspot VM thought:
bash-3.00$ java -help
Usage: java [-options] class [args...]
:
-client to select the "client" VM
-server to select the "server" VM
-hotspot is a synonym for the "client" VM [deprecated]
The default VM is server,
because you are running on a server-class machine.
So appears it thinks my new dual core Sony Vaio with 2 Gig Ram is a
server :( This is due to the joys of Smart tuning introduced in J2SE
5.0, which I''m sure is a good thing but perhaps the Automatic Server
detection needs revisiting, now that desktop machines are getting a lot
beefier :)
If I run [note the -client directive]:
$ java -client -jar /usr/java/demo/jfc/SwingSet2/SwingSet2.jar
I now get over 1800 shared classes loaded as reported by the dtrace
probe above. Unfortunately I now seem to crash even more often. Has the
client VM option been as heavily tested on nevada as the default server
VM option?
JR
John Rice
2007-Feb-25 21:21 UTC
[dtrace-discuss] Hotspot provider system crashes, class data sharing and automatic server detection gone wrong
I''ve been playing with this and it seems that the combination of using -Z and hotspot* makes the system hang a lot. I wanted to use these options so I could have the probe running before I started up different applications. Sticking to no -Z and hotspot$target seems to be a bit more stable. Curious if I''m the only one seeing this? Is it a nevada 56 issue? JR John Rice wrote:> Hi, > > I''ve been looking at the hotspot provider and wanted to demo some of > it''s features. Running on nevada 56 I''m getting a lot of system hangs > :( The system was rock solid until I started playing with the hotspot > provider, it seems particularly unhappy about the class-loaded probe > for some reason, though it''s hung with other method-entry probes. > Anyone else seeing this? > > The following script was run, as I wanted to demonstrate Class data > sharing. > > $ dtrace -Zqn ''hotspot*:::class-loaded{ @loaded[arg2, copyinstr(arg0, > arg1), arg3? "Shared":"Unique"] = count(); > @total[arg3?"Shared":"Unique"] = count();} END{printa("%-8d %40s %-10s > %8 at d\n", @loaded); printa("%-20s %@d\n", at total);}'' > > $ java -jar /usr/java/demo/jfc/SwingSet2/SwingSet2.jar > > Strangely all the classes where unique. Checked to see if I had a > class archive, I do: > /usr/jdk/instances/jdk1.6.0/jre/lib/i386/client/classes.jsa > > Then looked to see what the hotspot VM thought: > bash-3.00$ java -help > Usage: java [-options] class [args...] > : > -client to select the "client" VM > -server to select the "server" VM > -hotspot is a synonym for the "client" VM [deprecated] > The default VM is server, > because you are running on a server-class machine. > > So appears it thinks my new dual core Sony Vaio with 2 Gig Ram is a > server :( This is due to the joys of Smart tuning introduced in J2SE > 5.0, which I''m sure is a good thing but perhaps the Automatic Server > detection needs revisiting, now that desktop machines are getting a > lot beefier :) > > If I run [note the -client directive]: > $ java -client -jar /usr/java/demo/jfc/SwingSet2/SwingSet2.jar > > I now get over 1800 shared classes loaded as reported by the dtrace > probe above. Unfortunately I now seem to crash even more often. Has > the client VM option been as heavily tested on nevada as the default > server VM option? > > JR > _______________________________________________ > dtrace-discuss mailing list > dtrace-discuss at opensolaris.org
James C. McPherson
2007-Feb-25 22:00 UTC
[dtrace-discuss] Hotspot provider system crashes, class data sharing and automatic server detection gone wrong
Hi John, John Rice wrote:> I''ve been playing with this and it seems that the combination of using > -Z and hotspot* makes the system hang a lot. I wanted to use these > options so I could have the probe running before I started up different > applications. Sticking to no -Z and hotspot$target seems to be a bit > more stable. > Curious if I''m the only one seeing this? Is it a nevada 56 issue?I''m running snv_57 on a dual-core opteron. I''m using java "1.6.0-b105">> I''ve been looking at the hotspot provider and wanted to demo some of >> it''s features. Running on nevada 56 I''m getting a lot of system hangs >> :( The system was rock solid until I started playing with the hotspot >> provider, it seems particularly unhappy about the class-loaded probe >> for some reason, though it''s hung with other method-entry probes. >> Anyone else seeing this? >> >> The following script was run, as I wanted to demonstrate Class data >> sharing. >> >> $ dtrace -Zqn ''hotspot*:::class-loaded{ @loaded[arg2, copyinstr(arg0, >> arg1), arg3? "Shared":"Unique"] = count(); >> @total[arg3?"Shared":"Unique"] = count();} END{printa("%-8d %40s %-10s >> %8 at d\n", @loaded); printa("%-20s %@d\n", at total);}'' >> $ java -jar /usr/java/demo/jfc/SwingSet2/SwingSet2.jarthe D above gave me ^C 0 java/lang/Object Unique 1 Unique 1 and the jar died with this collection of stack traces: core ''core'' of 10516: java -jar /usr/java/demo/jfc/SwingSet2/SwingSet2.jar ----------------- lwp# 1 / thread# 1 -------------------- feef8d37 __lwp_wait (3, 8046550) + 7 feef4b43 _thrp_join (3, 0, 80465a0, 1) + 5d feef4c83 thr_join (3, 0, 80465a0) + 23 08058a47 ContinueInNewThread (8052260, 50000, 0, 8046e48, 8046e00) + 4f 08051f59 main (0, 806af9c, 8046ed4) + aa1 0805142a _start (3, 8047050, 8047055, 804705a, 0, 8047085) + 7a ----------------- lwp# 2 / thread# 2 -------------------- feef8a07 __pollsys (fe31bd40, 2, 0, 0) + 7 feeb41fe pselect (5, fe31be20, fef32260, fef32260, 0, 0) + 19e feeb450e select (5, fe31be20, 0, 0, 0) + 7e fe515fb0 SplashEventLoop (fe551f08) + 11c fe516453 SplashScreenThread (fe551f08) + df feef7d82 _thr_setup (fee42400) + 52 feef7fe0 _lwp_start (fee42400, 0, 0, 0, 0, 0) ----------------- lwp# 3 / thread# 3 -------------------- feeed7f4 sigacthandler (b, fe0ad744, fe0ad544) + 14 --- called from signal handler with signal 11 (SIGSEGV) --- feef1188 mutex_lock_impl (fef316e0, 0) + 18 feef13a2 mutex_lock (fef316e0) + 1d feea4de4 malloc (b) + 2c fe8aca9f __1cCosGmalloc6FI_pv_ (b) + 2b fea60c5e __1cLClassLoaderLadd_package6Fpkci_b_ (80f7cc8, 1) + 62 fe8ec0bc __1cLClassLoaderOload_classfile6FnMsymbolHandle_pnGThread__nTinstanceKlassHandle__ (fe0ada30, fee22da4) + 25c fe8e1b70 __1cQSystemDictionaryTload_instance_class6FnMsymbolHandle_nGHandle_pnGThread__nTinstanceKlassHandle__ (fe0adaf8, fee22da4, 0) + 314 fe8af3ac __1cQSystemDictionarybEresolve_instance_class_or_null6FnMsymbolHandle_nGHandle_2pnGThread__pnMklassOopDesc__ (fee22da4, 0, 0, 80f7800) + 514 fe8aa9db __1cQSystemDictionaryPresolve_or_null6FnMsymbolHandle_nGHandle_2pnGThread__pnMklassOopDesc__ (fee22da4, 0, 0, 80f7800) + 63 fec95002 __1cQSystemDictionaryPresolve_or_fail6FnMsymbolHandle_nGHandle_2bpnGThread__pnMklassOopDesc__ (fee22da4, 0, 0, 1, 80f7800) + 2a fec951fa __1cQSystemDictionaryPresolve_or_fail6FnMsymbolHandle_bpnGThread__pnMklassOopDesc__ (fee22da4, 1, 80f7800) + 32 fe9db82a __1cQSystemDictionarybCinitialize_preloaded_classes6FpnGThread__v_ (80f7800) + 36 fe9dc26b __1cQSystemDictionaryKinitialize6FpnGThread__v_ (80f7800) + c7 fe99ec6a __1cIUniverseHgenesis6FpnGThread__v_ (80f7800) + afa fe9a0080 __1cOuniverse2_init6F_v_ (fedde000, 80f7800, 80f7800, fe0adec4, feca0b3c, fee0dff8) + 30 fe9b5134 __1cMinit_globals6F_i_ (fee0dff8, fe0adf40, fedde000, fe0adec4, 1, 0) + 70 feca0b3c __1cHThreadsJcreate_vm6FpnOJavaVMInitArgs_pb_i_ (fe0adf40, fe0adf04) + 1c4 fe9bd412 JNI_CreateJavaVM (fe0adfc4, fe0adfc8, fe0adf40) + a6 080523bf JavaMain (8046e48) + 15f feef7d82 _thr_setup (fe040000) + 52 feef7fe0 _lwp_start (fe040000, 0, 0, 0, 0, 0) ----------------- lwp# 4 / thread# 4 -------------------- feef8dd7 ___lwp_cond_wait (80fdef8, 80fdee0) + 7 fec3621a __1cHMonitorEwait6Mbl_b_ (80fde88, 1, 0) + 46a fe8aab8d __1cNGCTaskManagerIget_task6MI_pnGGCTask__ (80fde48, 0) + 89 fe99acb1 __1cMGCTaskThreadDrun6M_v_ (80fe000) + 16d fec40007 java_start (80fe000) + d3 feef7d82 _thr_setup (fe040400) + 52 feef7fe0 _lwp_start (fe040400, 0, 0, 0, 0, 0) ----------------- lwp# 5 / thread# 5 -------------------- feef8dd7 ___lwp_cond_wait (80fdef8, 80fdee0) + 7 fec3621a __1cHMonitorEwait6Mbl_b_ (80fde88, 1, 0) + 46a fe8aab8d __1cNGCTaskManagerIget_task6MI_pnGGCTask__ (80fde48, 1) + 89 fe99acb1 __1cMGCTaskThreadDrun6M_v_ (80ff000) + 16d fec40007 java_start (80ff000) + d3 feef7d82 _thr_setup (fe040800) + 52 feef7fe0 _lwp_start (fe040800, 0, 0, 0, 0, 0) Running -hotspot gave this thread stack in the core (and no splash screen) ----------------- lwp# 3 / thread# 3 -------------------- feeed7f4 sigacthandler (b, fe28d6a4, fe28d4a4) + 14 --- called from signal handler with signal 11 (SIGSEGV) --- feef1188 mutex_lock_impl (fef316e0, 0) + 18 feef13a2 mutex_lock (fef316e0) + 1d feea5940 free (8110f38) + 20 fe83bfeb __1cCosEfree6Fpv_v_ (8110f38) + 1b fe8d624f __1cICHeapObj2k6Fpv_v_ (8110f38) + 1b feb1d12e __1cQPlaceholderEntrySremove_seen_thread6MpnGThread_nQPlaceholderTablePclassloadAction__b_ (810a820, 80f8000, 1) + 7a fe881a8d __1cQSystemDictionarybEresolve_instance_class_or_null6FnMsymbolHandle_nGHandle_2pnGThread__pnMklassOopDesc__ (80f8874, 0, 0, 80f8000) + 649 fe88143b __1cQSystemDictionaryPresolve_or_null6FnMsymbolHandle_nGHandle_2pnGThread__pnMklassOopDesc__ (80f8874, 0, 0, 80f8000) + 63 feb1be23 __1cQSystemDictionaryVresolve_super_or_fail6FnMsymbolHandle_1nGHandle_2bpnGThread__pnMklassOopDesc__ (80f8864, 80f8874, 0, 0, 0, 80f8000) + 16b fe8827fa __1cQSystemDictionaryRload_shared_class6FnTinstanceKlassHandle_nGHandle_pnGThread__1_ (fe28d9c8, 80f8860, 0) + 20a fe882578 __1cQSystemDictionaryRload_shared_class6FnMsymbolHandle_nGHandle_pnGThread__nTinstanceKlassHandle__ (fe28da2c, fec56bf4, 0) + 6c fe8824a6 __1cQSystemDictionaryTload_instance_class6FnMsymbolHandle_nGHandle_pnGThread__nTinstanceKlassHandle__ (fe28daf8, fec56bf4, 0) + 2fa fe881958 __1cQSystemDictionarybEresolve_instance_class_or_null6FnMsymbolHandle_nGHandle_2pnGThread__pnMklassOopDesc__ (fec56bf4, 0, 0, 80f8000) + 514 fe88143b __1cQSystemDictionaryPresolve_or_null6FnMsymbolHandle_nGHandle_2pnGThread__pnMklassOopDesc__ (fec56bf4, 0, 0, 80f8000) + 63 feb1baba __1cQSystemDictionaryPresolve_or_fail6FnMsymbolHandle_nGHandle_2bpnGThread__pnMklassOopDesc__ (fec56bf4, 0, 0, 1, 80f8000) + 2a feb1bcb2 __1cQSystemDictionaryPresolve_or_fail6FnMsymbolHandle_bpnGThread__pnMklassOopDesc__ (fec56bf4, 1, 80f8000) + 32 fe880a86 __1cQSystemDictionarybCinitialize_preloaded_classes6FpnGThread__v_ (80f8000) + 5e fe880967 __1cQSystemDictionaryKinitialize6FpnGThread__v_ (80f8000) + c7 fe875d2a __1cIUniverseHgenesis6FpnGThread__v_ (80f8000) + afa fe8751a4 __1cOuniverse2_init6F_v_ (fec32000, 80f8000, 80f8000, fe28dec4, feb26df0, fec44dc8) + 30 fe840d8c __1cMinit_globals6F_i_ (fec44dc8, fe28df40, fec32000, fe28dec4, 1, 0) + 70 feb26df0 __1cHThreadsJcreate_vm6FpnOJavaVMInitArgs_pb_i_ (fe28df40, fe28df04) + 1c4 fe83a11e JNI_CreateJavaVM (fe28dfc4, fe28dfc8, fe28df40) + a6 080523bf JavaMain (8046e38) + 15f feef7d82 _thr_setup (fe220000) + 52 feef7fe0 _lwp_start (fe220000, 0, 0, 0, 0, 0) and ^C 0 java/io/Serializable Shared 1 0 java/lang/Object Shared 1 Shared 2 I can''t run that demo with java 1.5.0. cheers, James C. McPherson -- Solaris kernel software engineer, system admin and troubleshooter http://www.jmcp.homeunix.com/blog Find me on LinkedIn @ http://www.linkedin.com/in/jamescmcpherson
Ekaterina Pavlova
2007-Feb-26 06:54 UTC
[dtrace-discuss] Hotspot provider system crashes, class data sharing and automatic server detection gone wrong
Hi John, I also observed the similar hangs on nevada 41 (but I am not sure it is related to -Z). We upgraded to nevada 58 and seems there are no more hangs. -katya John Rice wrote:> I''ve been playing with this and it seems that the combination of using > -Z and hotspot* makes the system hang a lot. I wanted to use these > options so I could have the probe running before I started up different > applications. Sticking to no -Z and hotspot$target seems to be a bit > more stable. > > Curious if I''m the only one seeing this? Is it a nevada 56 issue? > JR > > > John Rice wrote: > >> Hi, >> >> I''ve been looking at the hotspot provider and wanted to demo some of >> it''s features. Running on nevada 56 I''m getting a lot of system hangs >> :( The system was rock solid until I started playing with the hotspot >> provider, it seems particularly unhappy about the class-loaded probe >> for some reason, though it''s hung with other method-entry probes. >> Anyone else seeing this? >> >> The following script was run, as I wanted to demonstrate Class data >> sharing. >> >> $ dtrace -Zqn ''hotspot*:::class-loaded{ @loaded[arg2, copyinstr(arg0, >> arg1), arg3? "Shared":"Unique"] = count(); >> @total[arg3?"Shared":"Unique"] = count();} END{printa("%-8d %40s %-10s >> %8 at d\n", @loaded); printa("%-20s %@d\n", at total);}'' >> >> $ java -jar /usr/java/demo/jfc/SwingSet2/SwingSet2.jar >> >> Strangely all the classes where unique. Checked to see if I had a >> class archive, I do: >> /usr/jdk/instances/jdk1.6.0/jre/lib/i386/client/classes.jsa >> >> Then looked to see what the hotspot VM thought: >> bash-3.00$ java -help >> Usage: java [-options] class [args...] >> : >> -client to select the "client" VM >> -server to select the "server" VM >> -hotspot is a synonym for the "client" VM [deprecated] >> The default VM is server, >> because you are running on a server-class machine. >> >> So appears it thinks my new dual core Sony Vaio with 2 Gig Ram is a >> server :( This is due to the joys of Smart tuning introduced in J2SE >> 5.0, which I''m sure is a good thing but perhaps the Automatic Server >> detection needs revisiting, now that desktop machines are getting a >> lot beefier :) >> >> If I run [note the -client directive]: >> $ java -client -jar /usr/java/demo/jfc/SwingSet2/SwingSet2.jar >> >> I now get over 1800 shared classes loaded as reported by the dtrace >> probe above. Unfortunately I now seem to crash even more often. Has >> the client VM option been as heavily tested on nevada as the default >> server VM option? >> >> JR >> _______________________________________________ >> dtrace-discuss mailing list >> dtrace-discuss at opensolaris.org > > > _______________________________________________ > dtrace-discuss mailing list > dtrace-discuss at opensolaris.org-- Ekaterina Pavlova, VMSQE Team, St. Petersburg. http://blogs.sun.com/vmrobot/ http://blogs.sun.com/theaquarium_ru/
John Rice
2007-Feb-26 14:32 UTC
[dtrace-discuss] Hotspot provider system crashes, class data sharing and automatic server detection gone wrong
Thanks Jim and Ekaterina - too late now for me to risk an upgrade as I''m travelling today and will do the demo tomorrow :) So think I will stick to describing the feature and demo the method-entry stuff instead for this part of the talk. I''ll upgrade to 58 when I get home and post up any findings. JR Ekaterina Pavlova wrote:> Hi John, > > I also observed the similar hangs on nevada 41 > (but I am not sure it is related to -Z). > We upgraded to nevada 58 and seems there are no more hangs. > > -katya > > John Rice wrote: >> I''ve been playing with this and it seems that the combination of >> using -Z and hotspot* makes the system hang a lot. I wanted to use >> these options so I could have the probe running before I started up >> different applications. Sticking to no -Z and hotspot$target seems to >> be a bit more stable. >> >> Curious if I''m the only one seeing this? Is it a nevada 56 issue? >> JR >> >> >> John Rice wrote: >> >>> Hi, >>> >>> I''ve been looking at the hotspot provider and wanted to demo some of >>> it''s features. Running on nevada 56 I''m getting a lot of system >>> hangs :( The system was rock solid until I started playing with the >>> hotspot provider, it seems particularly unhappy about the >>> class-loaded probe for some reason, though it''s hung with other >>> method-entry probes. Anyone else seeing this? >>> >>> The following script was run, as I wanted to demonstrate Class data >>> sharing. >>> >>> $ dtrace -Zqn ''hotspot*:::class-loaded{ @loaded[arg2, >>> copyinstr(arg0, arg1), arg3? "Shared":"Unique"] = count(); >>> @total[arg3?"Shared":"Unique"] = count();} END{printa("%-8d %40s >>> %-10s %8 at d\n", @loaded); printa("%-20s %@d\n", at total);}'' >>> >>> $ java -jar /usr/java/demo/jfc/SwingSet2/SwingSet2.jar >>> >>> Strangely all the classes where unique. Checked to see if I had a >>> class archive, I do: >>> /usr/jdk/instances/jdk1.6.0/jre/lib/i386/client/classes.jsa >>> >>> Then looked to see what the hotspot VM thought: >>> bash-3.00$ java -help >>> Usage: java [-options] class [args...] >>> : >>> -client to select the "client" VM >>> -server to select the "server" VM >>> -hotspot is a synonym for the "client" VM [deprecated] >>> The default VM is server, >>> because you are running on a server-class machine. >>> >>> So appears it thinks my new dual core Sony Vaio with 2 Gig Ram is a >>> server :( This is due to the joys of Smart tuning introduced in J2SE >>> 5.0, which I''m sure is a good thing but perhaps the Automatic Server >>> detection needs revisiting, now that desktop machines are getting a >>> lot beefier :) >>> >>> If I run [note the -client directive]: >>> $ java -client -jar /usr/java/demo/jfc/SwingSet2/SwingSet2.jar >>> >>> I now get over 1800 shared classes loaded as reported by the dtrace >>> probe above. Unfortunately I now seem to crash even more often. Has >>> the client VM option been as heavily tested on nevada as the default >>> server VM option? >>> >>> JR >>> _______________________________________________ >>> dtrace-discuss mailing list >>> dtrace-discuss at opensolaris.org >> >> >> _______________________________________________ >> dtrace-discuss mailing list >> dtrace-discuss at opensolaris.org >