Ext3 users - Dec 2006 - maintain 6TB filesystem + fsck

i posted on rhel list about proper creating of 6tb ext3 filesystem and
tuning
here.......http://www.redhat.com/archives/nahant-list/2006-November/msg00239.html
i am reading lots of ext3 links like......
http://www.redhat.com/support/wpapers/redhat/ext3/
http://lists.centos.org/pipermail/centos/2005-September/052533.html
http://batleth.sapienti-sat.org/projects/FAQs/ext3-faq.html
............but not enough links for large TB arrays and ext3 :( there
is lots of old faq and information so pls show me errors of my ways
lol. after reading mailing list posts i have created filesystems like
this........
mkfs.ext3 -b 4096 -i 65536 -j -m 1 -O dir_index -L /prodspace1 /dev/sda1

i put output of mkfs.ext3 and tune2fs -l below. is there any thing
that i am mistaken about?? my other problem is fsck. i read
here.....http://listman.redhat.com/archives/ext3-users/2006-October/msg00005.html
'The major problem at this point is e2fsck time, which is about 1h/TB
for fast disks, at minimum (i.e. no major corruption found).'
.........is that ext3 or ext4? i don't know how long fsck will take w/
6TB ext3 filesystem. i first choose to disable auto fsck with 'tune2fs
-i0 -c0 /dev/sda1' but seems dangerous if filesystem become corrupt
without my knowledge! what is good balance betwen using auto fsck
after number of mounts or time pass and keeping fsck time short for
large arrays? info......
os is rhel es 4 update 4 w/ generic server hardware
storage hardware is multiple apple xserve raid w/ 6TB array each
filesystem expected to contain 10 mb files to maybe even 50 mb + 100mb

# tune2fs -l /dev/sda1
tune2fs 1.35 (28-Feb-2004)
Filesystem volume name:   /prodspace1
Last mounted on:          <not available>
Filesystem UUID:          7dccbede-5f4a-4cdf-b81e-129d3dd40106
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal resize_inode dir_index filetype
needs_recovery sparse_super large_file
Default mount options:    (none)
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              106733568
Block count:              1707722743
Reserved block count:     17077227
Free blocks:              1704243353
Free inodes:              106733557
First block:              0
Block size:               4096
Fragment size:            4096
Reserved GDT blocks:      1024
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         2048
Inode blocks per group:   64
Filesystem created:       Thu Nov 30 18:06:45 2006
Last mount time:          Thu Nov 30 18:26:11 2006
Last write time:          Thu Nov 30 18:26:11 2006
Mount count:              1
Maximum mount count:      38
Last checked:             Thu Nov 30 18:06:45 2006
Check interval:           15552000 (6 months)
Next check after:         Tue May 29 19:06:45 2007
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               128
Journal inode:            8
Default directory hash:   tea
Directory Hash Seed:      9ef5bfcf-74fd-49a1-b2dc-88aa9b881bd9
Journal backup:           inode blocks


# mkfs.ext3 -b 4096 -i 65536 -j -m 1 -O dir_index -L /prodspace1 /dev/sda1
mke2fs 1.35 (28-Feb-2004)
Filesystem label=/prodspace1
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
106733568 inodes, 1707722743 blocks
17077227 blocks (1.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=1711276032
52116 block groups
32768 blocks per group, 32768 fragments per group
2048 inodes per group
Superblock backups stored on blocks:
       32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
       4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968,
       102400000, 214990848, 512000000, 550731776, 644972544

Writing inode tables: done
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 38 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.

On Nov 30, 2006  21:55 -0500, Joseph Cheng wrote:
> http://listman.redhat.com/archives/ext3-users/2006-October/msg00005.html
> 'The major problem at this point is e2fsck time, which is about 1h/TB
> for fast disks, at minimum (i.e. no major corruption found).'
> .........is that ext3 or ext4?
I don't think it really matters.
> i don't know how long fsck will take w/ 6TB ext3 filesystem.
You have such a filesystem, test it...
> i first choose to disable auto fsck with 'tune2fs
> -i0 -c0 /dev/sda1' but seems dangerous if filesystem become corrupt
> without my knowledge! what is good balance betwen using auto fsck
> after number of mounts or time pass and keeping fsck time short for
> large arrays? info......
You can optionally run e2fsck -fn on a relatively quiet (though mounted)
filesystem, and if it checks (relatively) clean then you could reset the
fsck time in the superblock via tune2fs.
> filesystem expected to contain 10 mb files to maybe even 50 mb + 100mb
One of the major slowdowns for e2fsck is the number of inodes, so if you
expect to have very large files you should create the filesystem with
this in mind (i.e. "mke2fs -t largefile" or "mke2fs -t
largefile4").
Expect e2fsck RAM usage to be about .75 * num_inodes + .25 * num_blocks,
so in the neighbourhood of 500MB for your filesystem, so reducing inode
count would also help this a fair amount.

We are working on a patch to e2fsck and the kernel to allow e2fsck to
skip unused inodes/bitmaps in each group so that e2fsck time is improved.
It isn't quite ready for prime time, but has previously been discussed
in linux-ext4 in relation to the UNINIT flags in recent e2fsprogs.  It
would at least reduce e2fsck time to O(used_inodes) from O(total_inodes)
and also potentially avoid a lot of seeky IO.

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.