Hi, we have a 2 node / 3 node RAC installation with OCFS. We have the following observations. 1. TIME STAMP Issue We have noticed that the time stamp which is shown on the datafiles doesnt remain the same even after a shutdown normal /shutdown immediate i.e If I shutdown all RAC instances ( A , B , C) using shutdown normal / immediate, the timestamp on the datafiles are not the same. Even If I just bring up one node ( Say A ) and shut it down normally, after doing an Alter system checkpoint, I still dont see same time stamp on all the datafiles. ( I did an alter system checkpoint on a non-rac database and I noticed that all the datafiles changed the time stamp to the current timestamp) Is this expected behaviour from a RAC database on OCFS. ? 2. COPY / FTP Issue We tried to clone from a RAC to a NON-RAC database using a cold backup of the RAC Instance. On once occasion we used simple copy command (cp) to copy the files over to ext3 disk On the second occasion we used ftp command in bin mode to ftp the files over to the destination node to ext3 disk. On both occassions while creating the new control file on Target node, we got error stating the System file needs more recovery. ( Althoug we are 100% sure that on both occassions databases were shutdown normally for the backup) On both above mentioned ocassions we were able to correct the problem using cp --o_direct command to copy the files over. When we used cp --o_direct , we were able to create the Target Database successfully. Has anyone noticed any similar issue with copy / ftp. ? R'gds Varghese Abraham.
> 1. TIME STAMP Issue > Is this expected behaviour from a RAC database on OCFS. ?yes timestamps don't matter to run oracle and it's an operation that needs to sync up to nodes so it would hurt performance in a cluster. we know it could be useful to have but in general providing the functionality hurts more than what it gives you in return.> 2. COPY / FTP Issue > > Has anyone noticed any similar issue with copy / ftp. ?you didn't mention which version you use. either way, using directio would be the ideal it depends on which node you were copying from etc, the buffers might still be stale in the cache or so, if you do directio that is not a problem
> 1. TIME STAMP IssueOCFS changes the mtime only if the file meta-data is changed. This includes a extend/truncate of the file or a simple touch. Other local filesystems update mtime during every write. As ocfs is a clustered system, updating time or any meta-data changes, requires a cluster wide lock, which is an expensive operation. So, we avoid that during writes. So, this has nothing to do with RAC.... just a filesystem behavior.> 2. COPY / FTP IssueThat's why we provide the o_direct enabled fileutils. As this is a clustered fs, the data blocks are being updated by different nodes. To ensure a consistent view of the file, one should always use the o_direct flag inorder to bypass the possible stale data in that node's buffer cache. On Mon, 2004-08-30 at 12:50, Varghese Abraham wrote:> Hi, > we have a 2 node / 3 node RAC installation with OCFS. > We have the following observations. > > > 1. TIME STAMP Issue > > We have noticed that the time stamp which is shown on the datafiles doesnt remain the same even after a shutdown normal /shutdown immediate > > i.e If I shutdown all RAC instances ( A , B , C) using shutdown normal / immediate, the timestamp on the datafiles are not the same. > Even If I just bring up one node ( Say A ) and shut it down normally, after doing an Alter system checkpoint, > I still dont see same time stamp on all the datafiles. > > ( I did an alter system checkpoint on a non-rac database and I noticed that all the datafiles changed the time stamp to the current timestamp) > > Is this expected behaviour from a RAC database on OCFS. ? > > > 2. COPY / FTP Issue > > We tried to clone from a RAC to a NON-RAC database using a cold backup of the RAC Instance. > On once occasion we used simple copy command (cp) to copy the files over to ext3 disk > On the second occasion we used ftp command in bin mode to ftp the files over to the destination node to ext3 disk. > > On both occassions while creating the new control file on Target node, we got error stating the System file needs more recovery. ( Althoug we are 100% sure that on both occassions databases were shutdown normally for the backup) > > On both above mentioned ocassions we were able to correct the problem using cp --o_direct command to copy the files over. > When we used cp --o_direct , we were able to create the Target Database successfully. > > Has anyone noticed any similar issue with copy / ftp. ? > > > R'gds > Varghese Abraham. > _______________________________________________ > Ocfs-users mailing list > Ocfs-users@oss.oracle.com > http://oss.oracle.com/mailman/listinfo/ocfs-users
Hi Sunil , Wim Thanks for your inputs. We are on Linux AS 2.1 kernel e38 OCFS version ---------------------------------- [root@usingorcdb03 root]# rpm -qa | grep ocfs ocfs-tools-1.0.10-1 ocfs-2.4.9-e-enterprise-1.0.12-1 ocfs-support-1.0.10-1 [root@usingorcdb03 root]# -------------------- Regarding point 2, can you clarify a bit further. What I understand from your / wim's mail is that data blocks would be present somewhere in memory buffer and when we copy / ftp , those Memory blocks are getting used. Is my understanding right. ? ( Should this be the case even when the database is shutdown , that Linux would use memory buffers to do a copy /ftp. ) R'gds Varghese Abraham. -----Original Message----- From: Sunil Mushran [mailto:Sunil.Mushran@oracle.com] Sent: Monday, August 30, 2004 1:25 PM To: Varghese Abraham Cc: ocfs-users@oss.oracle.com Subject: Re: [Ocfs-users] Observations> 1. TIME STAMP IssueOCFS changes the mtime only if the file meta-data is changed. This includes a extend/truncate of the file or a simple touch. Other local filesystems update mtime during every write. As ocfs is a clustered system, updating time or any meta-data changes, requires a cluster wide lock, which is an expensive operation. So, we avoid that during writes. So, this has nothing to do with RAC.... just a filesystem behavior.> 2. COPY / FTP IssueThat's why we provide the o_direct enabled fileutils. As this is a clustered fs, the data blocks are being updated by different nodes. To ensure a consistent view of the file, one should always use the o_direct flag inorder to bypass the possible stale data in that node's buffer cache. On Mon, 2004-08-30 at 12:50, Varghese Abraham wrote:> Hi, > we have a 2 node / 3 node RAC installation with OCFS. > We have the following observations. > > > 1. TIME STAMP Issue > > We have noticed that the time stamp which is shown on the datafiles doesnt remain the same even after a shutdown normal /shutdown immediate > > i.e If I shutdown all RAC instances ( A , B , C) using shutdown normal / immediate, the timestamp on the datafiles are not the same. > Even If I just bring up one node ( Say A ) and shut it down normally, after doing an Alter system checkpoint, > I still dont see same time stamp on all the datafiles. > > ( I did an alter system checkpoint on a non-rac database and I noticed that all the datafiles changed the time stamp to the current timestamp) > > Is this expected behaviour from a RAC database on OCFS. ? > > > 2. COPY / FTP Issue > > We tried to clone from a RAC to a NON-RAC database using a cold backup of the RAC Instance. > On once occasion we used simple copy command (cp) to copy the files over to ext3 disk > On the second occasion we used ftp command in bin mode to ftp the files over to the destination node to ext3 disk. > > On both occassions while creating the new control file on Target node, we got error stating the System file needs more recovery. ( Althoug we are 100% sure that on both occassions databases were shutdown normally for the backup) > > On both above mentioned ocassions we were able to correct the problem using cp --o_direct command to copy the files over. > When we used cp --o_direct , we were able to create the Target Database successfully. > > Has anyone noticed any similar issue with copy / ftp. ? > > > R'gds > Varghese Abraham. > _______________________________________________ > Ocfs-users mailing list > Ocfs-users@oss.oracle.com > http://oss.oracle.com/mailman/listinfo/ocfs-users