I have something like 300G I routinely backup. This includes some large 12Gig images and other files. I had been using ext3 on an external USB disk for part of the process. Under ext3 doing "rsync -a /home /mnt/external_back/backup.jun.27.2011" it took 200 minutes. I took the same computer, same external HD and reformatted it for ext4 (mkfs.ext4 /dev/sdd1). I then started the same rsync as above. The time was reduced to 170 minutes. This is on Centos 5.6 x86_64. I did notice the [ext4-dio-unwrit] process that starts up. One process for each core. I was concerned when they did not go away. But google says that is normal. :) Jerry
On Mon, Jun 27, 2011 at 2:10 PM, Jerry Geis <geisj at pagestation.com> wrote:> I have something like 300G I routinely backup. > This includes some large 12Gig images and other files. > > I had been using ext3 on an external USB disk for part of the process. > > Under ext3 doing "rsync -a /home /mnt/external_back/backup.jun.27.2011" > it took 200 minutes. > > I took the same computer, same external HD and reformatted it for ext4 > (mkfs.ext4 /dev/sdd1). > I then started the same rsync as above. The time was reduced to 170 minutes. > > This is on Centos 5.6 x86_64. > > I did notice the [ext4-dio-unwrit] process that starts up. One process > for each core. > I was concerned when they did not go away. But google says that is > normal. :) > > Jerry > _______________________________________________ > CentOS mailing list > CentOS at centos.org > http://lists.centos.org/mailman/listinfo/centos >Thanx for the feeback Jerry, What filesystem does the storage on the server use, as matter of interest? -- Kind Regards Rudi Ahlers SoftDux Website: http://www.SoftDux.com Technical Blog: http://Blog.SoftDux.com Office: 087 805 9573 Cell: 082 554 7532
> > Thanx for the feeback Jerry, > > > What filesystem does the storage on the server use, as matter of interest? > > >Sure, the server still has ext3. Jerry
On 6/27/2011 8:10 AM, Jerry Geis wrote:> I have something like 300G I routinely backup. > This includes some large 12Gig images and other files. > > I had been using ext3 on an external USB disk for part of the process. > > Under ext3 doing "rsync -a /home /mnt/external_back/backup.jun.27.2011" > it took 200 minutes. > > I took the same computer, same external HD and reformatted it for ext4 > (mkfs.ext4 /dev/sdd1). > I then started the same rsync as above. The time was reduced to 170 minutes. >A time reduction for using ext4 in that scenario does not surprise me. Under ext3, deleting large mult-gigabyte files requires a lot of activity as it tracks down and marks the blocks as free. In ext4, this process is a lot faster due to the use of extents to track which blocks are used by large files. Just the faster deletion times with ext4 might account for the time difference. Under ext3, deleting a 10GB file might take a minute or two, but it will only take a few seconds under ext4. (It's the primary reason that I started using ext4 last year for any shares / file systems where I needed to store large files.)