Hello I try again. Can anybody of you acknowledge or not if mode data=journal in ext3 is safe to use in Linux kernel 2.6.x? Wee need to have a very consistent and integrity for our filesystem, and it would then be desired to journal both data and metadata. But if this mode can corrupt the filesystem as both Phil White and Nicolas Kowalski has experienced, it may be more advised to use mode data=ordered instead. Data integrity is much more important for us than speed. What do you people out there say? I also try to post this in the kernel mailing list. I have not subscribed to the kml so if anybody there have som advisory about this I would be pleased if you could CC me. Petter On Mon, 2004-06-07 at 10:21, Petter Larsen wrote:> Hello > > I can see several postings on this mailing-list that people have > problem > with mounting ext3 partition with mode data=journal. > > See URL's: > https://www.redhat.com/archives/ext3-users/2004-March/msg00000.html > https://www.redhat.com/archives/ext3-users/2004-March/msg00050.html > > We are going to use ext3 on a Compact Flash disk in true IDE mode. We > need this filesystem to be as safe and consistent as possible. We can > not tolerate any garbage in the files after a crash or sudden power > failures. We have then decided to use ext3 with mode data=journal. > > Can I rely on this? > We use kernel 2.6.5 on PowerPC 8260, and may be using newer kernels > later in the project. > > > Best regards > -- > Petter Larsen > cand. scient. > moreCom as > 913 17 222 > > > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users-- Petter Larsen cand. scient. moreCom as 913 17 222
On Tue, 2004-06-15 at 20:09 +0200, Petter Larsen wrote:> Can anybody of you acknowledge or not if mode data=journal in ext3 is > safe to use in Linux kernel 2.6.x? > > Wee need to have a very consistent and integrity for our filesystem, and > it would then be desired to journal both data and metadata. > > But if this mode can corrupt the filesystem as both Phil White and > Nicolas Kowalski has experienced, it may be more advised to use mode > data=ordered instead. > > Data integrity is much more important for us than speed.I ran ext3 with data=journal on 2.6.6smp for about a week on a heavily loaded system (I mean it). I did not ever experience filesystem corruption (related to the fs code). I did, however, hit complete system lockup once. It *may* have been unrelated to the fs code. (If you use quota, it *will* lock. The author is working on a fix. Above, I am referring to a lockup with quota off). Eugene -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part URL: <http://listman.redhat.com/archives/ext3-users/attachments/20040615/17b901be/attachment.sig>
Petter Larsen wrote:> > Data integrity is much more important for us than speed. >You might want to consider ReiserFS or one of the others which were designed with journaling in mind. And I hope you're using RAID1 or RAID5.
On Tue, 15 Jun 2004, Petter Larsen wrote:> HelloHello.> Wee need to have a very consistent and integrity for our filesystem, and > it would then be desired to journal both data and metadata. > > But if this mode can corrupt the filesystem as both Phil White and > Nicolas Kowalski has experienced, it may be more advised to use mode > data=ordered instead.In my case, the filesystem on this (old, so perhaps with disks beginning to fail) server was not corrupted. I only got some strange error messages, which went away when I switched back the data mode to ordered, as Phil White told me to try. Best regards. -- Nicolas
Petter, I was never able to resolve the problems I had with data=journal with the 2.4 kernel. I did *not* try the 2.6 kernel though, so I can't give you any data points there. In the end, I settled for data=ordered, and have never seen the problems I described in my original posts. Also, to give you some background, I had been using ReiserFS before switching to ext3, and I experienced a lot of corruption with Reiser (my company makes linux based appliances which sometimes get turned off while under heavy IO). Since ReiserFS doesn't do data journalling (metadata only), we consistently ended up with corrupt files. After this, I decided to try ext3 with data=journal, and I never even got far enough with load testing to try the 'hard reset' test. It would consistently crash in the fs code under heavy load. We have since had no problems with data=ordered, and since it writes data blocks before writing metadata to the journal, we don't see corrupt files anymore (even on hard resets). If data integrity (within the file) is important to you in the face of a crash or power loss, do NOT use ReiserFS or ext3 data=writeback. If your application never overwrites data in files, you will be just fine using data=ordered (appending to files or creating new files is pretty much guaranteed to never cause corruption). If you need to overwrite data in files, you need to use data=journal (and probably beg people to fix it) or rewrite your application to use some other method (i.e. copy the file, delete the old one) and just use data=ordered. --Phil> > > Petter Larsen wrote: > >> >> Data integrity is much more important for us than speed. >> > > > You might want to consider ReiserFS or one of the others which were > designed with journaling in mind. And I hope you're using RAID1 or RAID5. > >
Hello I comment inline..> PL> Can anybody of you acknowledge or not if mode data=journal in ext3 is > PL> safe to use in Linux kernel 2.6.x? > PL> Wee need to have a very consistent and integrity for our filesystem, and > PL> it would then be desired to journal both data and metadata. > > OLEG> Actually data=journal mode would gain you mostly zero extra consistency compared > to data=ordered mode. (the only more consistency bit that you get is > correct mtime on files that have their pages overwritten, I think). > You have zero control over transaction boundaries in ext3, so you still need > to design your applications in such a way that they have their own > sort of transactions (if this is needed).So your conclusion is that data=journal mode is useless if you do not want a correct mtime? It would be a littles sense in developing the data=journal mode if this is the only benefit, don't you think?>From the Linux/Documentation/filesystems/ext3.txtdata=journal All data are committed into the journal prior to being written into the main file system. data=ordered (*) All data are forced directly out to the main file system prior to its metadata being committed to the journal. My problem is that ext3 in the latest kernel, 2.6.x and the latest 2.4.x, are not well documented around the web. Whitepapers and so are pretty old. Much have changed I belive in ext3 since it was first introduced by Dr. Tweedie. The first release was journaling both data and metadata, se also the transcript from Dr. Tweedie from the Ottawa Linux Symposium 20th July 2000. http://olstrans.sourceforge.net/release/OLS2000-ext3/OLS2000-ext3.html There he says that they are journaling both metadata and data, but that the design goal is not to do that. So can this be interpreted that mode data=journal is only there for historic reasons?> PL> Data integrity is much more important for us than speed. > > OLEG> It is not clear what sort of extra data integrity do you expect from data > journaling mode and why do you think it is there.I would belive that the goal for such a mode data=journal would gain extra data integrity because it also journals data. Why should it not? I would belive that it makes sense to have these different modes so people can choose the best mode for there applications.> OLEG> Garbage in files should not happen in data ordered mode as data pages are > written first before metadata updates are committed.Are you sure? Petter
> > > > Data integrity is much more important for us than speed. > > > > > You might want to consider ReiserFS or one of the others which were > designed with journaling in mind. And I hope you're using RAID1 or RAID5.We are using ext3 on a compact flash disk in an embedded device. So we are not using RAID systems. Best regards -- Petter Larsen cand. scient. moreCom as 913 17 222