Robert Milkowski
2010-Mar-26 12:51 UTC
[dtrace-discuss] dtrace pid provider crashes Oracle
Hi, I have an Oracle 8.1.7.4 64bit running in a Solaris 9 branded zone on SPARC. If I use dtrace on ora_ckpt proccess to trace libc:semop:entry Oracle crashes after I try to put a tablespace in a backup mode. The tablespace becomes corrupted as well. So here what happens in mode detail: First I start Oracle and open a database - so far so good. Then I double check if I can put a table in a hot backup mode and then put it back - works fine, I tried several times. So now lets start a dtrace from a global zone: # dtrace -n pid24394:libc.so.1:semop:entry dtrace: description ''pid24394:libc.so.1:semop:entry'' matched 1 probe And now lets put a tablespace in a hotbackup mode: SQL> alter tablespace TEMP begin backup; alter tablespace TEMP begin backup * ERROR at line 1: ORA-00470: LGWR process terminated with error SQL> and Oracle crashed. In the database alert.log I got: Completed: ALTER DATABASE OPEN Fri Mar 26 12:23:27 2010 Errors in file /i321/ua01/app/oracle/admin/I321/bdump/i321_ckpt_24394.trc: ORA-07445: exception encountered: core dump [0000000000000040] [SIGSEGV] [Address not mapped to object] [64] [] [] Fri Mar 26 12:23:27 2010 Errors in file /i321/ua01/app/oracle/admin/I321/bdump/i321_ckpt_24394.trc: ORA-07445: exception encountered: core dump [00000001008D617C] [SIGSEGV] [Address not mapped to object] [64] [] [] ORA-07445: exception encountered: core dump [0000000000000040] [SIGSEGV] [Address not mapped to object] [64] [] [] Fri Mar 26 12:23:28 2010 Errors in file /i321/ua01/app/oracle/admin/I321/bdump/i321_ckpt_24394.trc: ORA-07445: exception encountered: core dump [00000001008D617C] [SIGSEGV] [Address not mapped to object] [64] [] [] ORA-07445: exception encountered: core dump [00000001008D617C] [SIGSEGV] [Address not mapped to object] [64] [] [] ORA-07445: exception encountered: core dump [0000000000000040] [SIGSEGV] [Address not mapped to object] [64] [] [] Fri Mar 26 12:23:35 2010 alter tablespace TEMP begin backup Fri Mar 26 12:23:37 2010 ORA-470 signalled during: alter tablespace TEMP begin backup ... Fri Mar 26 12:24:26 2010 Errors in file /i321/ua01/app/oracle/admin/I321/bdump/i321_pmon_24388.trc: ORA-00470: LGWR process terminated with error Fri Mar 26 12:24:26 2010 PMON: terminating instance due to error 470 Instance terminated by PMON, pid = 24388 And the trace file referenced in alert log: Dump file /i321/ua01/app/oracle/admin/I321/bdump/i321_pmon_24388.trc Oracle8i Enterprise Edition Release 8.1.7.4.0 - 64bit Production With the Partitioning option JServer Release 8.1.7.4.0 - 64bit Production ORACLE_HOME = /i321/ua01/app/oracle/product/8.1.7 System name: SunOS Node name: mk-imscluster-imsdb Release: 5.9 Version: Generic_Virtual Machine: sun4u Instance name: I321 Redo thread mounted by this instance: 1 Oracle process number: 2 Unix process pid: 24388, image: oracle at mk-imscluster-imsdb (PMON) *** 2010-03-26 12:24:26.649 *** SESSION ID:(1.1) 2010-03-26 12:24:26.646 error 470 detected in background process ORA-00470: LGWR process terminated with error Now if I try to open the database again I will get an error: SQL> startup; ORACLE instance started. Total System Global Area 1.7636E+10 bytes Fixed Size 102076 bytes Variable Size 1528254464 bytes Database Buffers 1.6106E+10 bytes Redo Buffers 1040384 bytes Database mounted. ORA-01113: file 3 needs media recovery ORA-01110: data file 3: ''/IMSDBdata/oradata/I321/temp.dbf'' SQL> I have the core file if anyone is interested. The issue is fully reproducible.
To close this out: we''ve determined that the pid provider is not supported by Solaris 8 and 9 branded zones. The request has been logged to either disallow this explicitly or to introduce the needed support. Adam On Mar 26, 2010, at 5:51 AM, Robert Milkowski wrote:> Hi, > > I have an Oracle 8.1.7.4 64bit running in a Solaris 9 branded zone on SPARC. > If I use dtrace on ora_ckpt proccess to trace libc:semop:entry Oracle crashes after I try to put a tablespace in a backup mode. The tablespace becomes corrupted as well. > > > So here what happens in mode detail: > > > First I start Oracle and open a database - so far so good. > Then I double check if I can put a table in a hot backup mode and then put it back - works fine, I tried several times. > > So now lets start a dtrace from a global zone: > > # dtrace -n pid24394:libc.so.1:semop:entry > dtrace: description ''pid24394:libc.so.1:semop:entry'' matched 1 probe > > > And now lets put a tablespace in a hotbackup mode: > > SQL> alter tablespace TEMP begin backup; > alter tablespace TEMP begin backup > * > ERROR at line 1: > ORA-00470: LGWR process terminated with error > > > SQL> > > > and Oracle crashed. > > > In the database alert.log I got: > > Completed: ALTER DATABASE OPEN > Fri Mar 26 12:23:27 2010 > Errors in file /i321/ua01/app/oracle/admin/I321/bdump/i321_ckpt_24394.trc: > ORA-07445: exception encountered: core dump [0000000000000040] [SIGSEGV] [Address not mapped to object] [64] [] [] > Fri Mar 26 12:23:27 2010 > Errors in file /i321/ua01/app/oracle/admin/I321/bdump/i321_ckpt_24394.trc: > ORA-07445: exception encountered: core dump [00000001008D617C] [SIGSEGV] [Address not mapped to object] [64] [] [] > ORA-07445: exception encountered: core dump [0000000000000040] [SIGSEGV] [Address not mapped to object] [64] [] [] > Fri Mar 26 12:23:28 2010 > Errors in file /i321/ua01/app/oracle/admin/I321/bdump/i321_ckpt_24394.trc: > ORA-07445: exception encountered: core dump [00000001008D617C] [SIGSEGV] [Address not mapped to object] [64] [] [] > ORA-07445: exception encountered: core dump [00000001008D617C] [SIGSEGV] [Address not mapped to object] [64] [] [] > ORA-07445: exception encountered: core dump [0000000000000040] [SIGSEGV] [Address not mapped to object] [64] [] [] > Fri Mar 26 12:23:35 2010 > alter tablespace TEMP begin backup > Fri Mar 26 12:23:37 2010 > ORA-470 signalled during: alter tablespace TEMP begin backup > ... > Fri Mar 26 12:24:26 2010 > Errors in file /i321/ua01/app/oracle/admin/I321/bdump/i321_pmon_24388.trc: > ORA-00470: LGWR process terminated with error > Fri Mar 26 12:24:26 2010 > PMON: terminating instance due to error 470 > Instance terminated by PMON, pid = 24388 > > > And the trace file referenced in alert log: > > > Dump file /i321/ua01/app/oracle/admin/I321/bdump/i321_pmon_24388.trc > Oracle8i Enterprise Edition Release 8.1.7.4.0 - 64bit Production > With the Partitioning option > JServer Release 8.1.7.4.0 - 64bit Production > ORACLE_HOME = /i321/ua01/app/oracle/product/8.1.7 > System name: SunOS > Node name: mk-imscluster-imsdb > Release: 5.9 > Version: Generic_Virtual > Machine: sun4u > Instance name: I321 > Redo thread mounted by this instance: 1 > Oracle process number: 2 > Unix process pid: 24388, image: oracle at mk-imscluster-imsdb (PMON) > > *** 2010-03-26 12:24:26.649 > *** SESSION ID:(1.1) 2010-03-26 12:24:26.646 > error 470 detected in background process > ORA-00470: LGWR process terminated with error > > > > Now if I try to open the database again I will get an error: > > SQL> startup; > ORACLE instance started. > > Total System Global Area 1.7636E+10 bytes > Fixed Size 102076 bytes > Variable Size 1528254464 bytes > Database Buffers 1.6106E+10 bytes > Redo Buffers 1040384 bytes > Database mounted. > ORA-01113: file 3 needs media recovery > ORA-01110: data file 3: ''/IMSDBdata/oradata/I321/temp.dbf'' > > > SQL> > > > I have the core file if anyone is interested. > > The issue is fully reproducible. > _______________________________________________ > dtrace-discuss mailing list > dtrace-discuss at opensolaris.org-- Adam Leventhal, Fishworks http://blogs.sun.com/ahl