Has anyone experienced a significant degradation in ext3 performance when using it on a Multi-TeraByte RAID? As part of an experimental setup, I hooked up three 300GB drives and made an EXT3 RAID5 out of them, using the entire space one each drive, and started throwing a large number of files in the size-range 3KB to 50 KB. Then, I deleted the raid, and created a new one, but this time, I used only 3 Gigs from each drive (a very small RAID compared to the earlier one). After repeating the same test, a huge improvement in performance was see - hence, the question : does ext3 performance degrade significantly as the file system size increases? Thanks in Advance, MK --------------------------------- Do you Yahoo!? Yahoo! Movies - Buy advance tickets for 'Shrek 2' -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://listman.redhat.com/archives/ext3-users/attachments/20040512/395c2ec8/attachment.htm>
This can easily be explained by seek time. If you have a 3GB partition on a 300GB disk, you are only using 1% of the surface of your disk. During the test on a smaller partition, head doesn't have to move as far as it does with the larger partition. Charles On Wed, 2004-05-12 at 22:09, M K wrote:> Has anyone experienced a significant degradation in ext3 performance > when using it on a Multi-TeraByte RAID? As part of an experimental > setup, I hooked up three 300GB drives and made an EXT3 RAID5 out of > them, using the entire space one each drive, and started throwing a > large number of files in the size-range 3KB to 50 KB. Then, I deleted > the raid, and created a new one, but this time, I used only 3 Gigs > from each drive (a very small RAID compared to the earlier one). After > repeating the same test, a huge improvement in performance was see - > hence, the question : does ext3 performance degrade significantly as > the file system size increases? > > Thanks in Advance, > MK > > > ______________________________________________________________________ > Do you Yahoo!? > Yahoo! Movies - Buy advance tickets for 'Shrek 2' > > ______________________________________________________________________ > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users
On May 12, 2004 19:09 -0700, M K wrote:> Has anyone experienced a significant degradation in ext3 performance > when using it on a Multi-TeraByte RAID? As part of an experimental setup, > I hooked up three 300GB drives and made an EXT3 RAID5 out of them, using > the entire space one each drive, and started throwing a large number > of files in the size-range 3KB to 50 KB. Then, I deleted the raid, and > created a new one, but this time, I used only 3 Gigs from each drive (a > very small RAID compared to the earlier one). After repeating the same > test, a huge improvement in performance was see - hence, the question: > does ext3 performance degrade significantly as the file system size > increases?Are you using 2.4 or 2.6 kernels? In the 2.4 kernel files are allocated evenly across all of the filesystem space. However, for large filesystems this is not very effective. In 2.6 kernels the Orlov allocator will keep files created by the same process (e.g. tar) in a single group if possible, which localizes block allocation and avoids seeks. Cheers, Andreas -- Andreas Dilger http://sourceforge.net/projects/ext2resize/ http://www-mddsp.enel.ucalgary.ca/People/adilger/
On Thu, 13 May 2004, Andreas Dilger wrote:> Are you using 2.4 or 2.6 kernels? In the 2.4 kernel files are allocated > evenly across all of the filesystem space. However, for large > filesystems this is not very effective. In 2.6 kernels the Orlov > allocator will keepuh oh. I've been lurking on this list for some time, trying to learn as much about ext3 as possible. This looks bad. Can I ask for some advice? Say I had a 45gb disk, with a 30gb ext3 partition at the end on which I had some important stuff stored. I remove all partitions on the disk, create one large ext3 partition and begin filling the disk with data. I realise my mistake after about 8gb was written. Is it too late to restore about 300mb of JPG files from this disk? Do writes happen sequentially, or are they scattered all over the place? If they are scattered, would they have hosed the data completely or can parts be recovered? Luke.