Ext3 users - Sep 2003 - BIG Bug on Linux EXT3 file system, or "/bin/ls" problem?? Please Help!!!

Hi, all,

  I "found" a big bug either related with Linux Ext3 file system or
"bin/ls"
command. The problem is: 
File system is no more than 220GB, but "/bin/ls" reports holding
>370GB
data! The system is Redhat 8.0, with a Generic Kernel 2.4.20. Any
suggestions are greately appreciated. more information can be available upon
requests.

The following is the results:

sh-2.05b# cat /proc/version
Linux version 2.4.20-v5 (root@vista104.alexa.com) (gcc version 3.2 20020903
(Red Hat Linux 8.0 3.2-7)) #3 SMP Thu May 1 17:37:52 PDT 2003
sh-2.05b# ls -alF /bin/ls
-rwxr-xr-x    1 root     root        67884 Sep  2  2002 /bin/ls*
sh-2.05b# md5sum /bin/ls
d55769791bd4775b10febacd92552c2f  /bin/ls
sh-2.05b# rpm -qf /bin/ls
fileutils-4.1.9-11
sh-2.05b#


sh-2.05b# df -k /0
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/hda11           222653644 211340252      3232 100% /0

sh-2.05b# ls -alF /0
total 211233264
drwxrwxr-t    5 root     wheel        4096 Sep 17 23:35 ./
drwxr-xr-x   24 root     root         4096 Sep 17 22:57 ../
lrwxrwxrwx    1 root     wheel           4 May 16 19:55 .final -> ../0/
-rw-r--r--    1 3094     wheel          26 Aug 20 09:08 ando.idx
-rw-r--r--    1 3094     wheel    193750787024 Sep 16 00:05 ando103.idx.2
-rw-r--r--    1 3094     wheel    185336663408 Sep 15 07:55 ando104.idx.2
drwxr-xr-x    3 root     root         4096 Jun 25 14:32 export/
drwxrwxr-t    2 root     wheel       16384 May 16 19:41 lost+found/
drwxrwxrwt    2 root     wheel        4096 Sep 17 23:35 tmp/
sh-2.05b#

There are two big files above, and the total size is bigger then underlying
file system size!! What a shame!

But "df" reports reasonable size:

sh-2.05b# du -sb /0/ando103.idx.2
30784983040     /0/ando103.idx.2

sh-2.05b# du -sb /0/ando104.idx.2
185517842432    /0/ando104.idx.2
sh-2.05b#

see the sizes reported for ando103.idx.2 by "df" and "ls"
greatly
contradict! I tried to unmount the file system and run "e2fsck -y -f
/0" but
that does no help. problem still keeps there. Then I tried to run debugfs,
and the following is the result, hope the information will be helpful for
bug-shooting:

sh-2.05b# debugfs /dev/hda11
debugfs 1.27 (8-Mar-2002)
debugfs:  stats
Filesystem volume name:   /0
Last mounted on:          <not available>
Filesystem UUID:          7bda9ec2-cd33-470a-8a18-a4d812e9196d
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal filetype needs_recovery sparse_super
large
_file
Filesystem state:         clean
Errors behavior:          Continue
Filesystem OS type:       Linux
Inode count:              28278784
Block count:              56550800
Reserved block count:     2827540
Free blocks:              2828348
Free inodes:              28278747
First block:              0
Block size:               4096
Fragment size:            4096
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         16384
Inode blocks per group:   512
Last mount time:          Wed Sep 17 23:33:57 2003
Last write time:          Wed Sep 17 23:33:57 2003
Mount count:              1
Maximum mount count:      37
Last checked:             Wed Sep 17 23:33:42 2003
Check interval:           15552000 (6 months)
Next check after:         Mon Mar 15 22:33:42 2004
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               128
Journal UUID:             <none>
Journal inode:            8
Journal device:           0x0000
First orphan inode:       0
 Group  0: block bitmap at 15, inode bitmap at 16, inode table at 17
           0 free blocks, 16369 free inodes, 2 used directories
......

debugfs:  stat ando103.idx.2
Inode: 14   Type: regular    Mode:  0644   Flags: 0x0   Generation:
2615178425
User:  3094   Group:    10   Size: 193750787024
File ACL: 0    Directory ACL: 0
Links: 1   Blockcount: 60126920
Fragment:  Address: 0    Number: 0    Size: 0
ctime: 0x3f695229 -- Wed Sep 17 23:35:21 2003
atime: 0x3f676cab -- Tue Sep 16 13:03:55 2003
mtime: 0x3f66b61f -- Tue Sep 16 00:05:03 2003
BLOCKS:
(0):23330295, (1-11):23330818-23330828, (IND):23330829,
(12-14):23330830-2333083
2, (15-27):23332155-23332167, .....

debugfs:  stat ando104.idx.2
Inode: 13   Type: regular    Mode:  0644   Flags: 0x0   Generation:
2615178405
User:  3094   Group:    10   Size: 185336663408
File ACL: 0    Directory ACL: 0
Links: 1   Blockcount: 362339536
Fragment:  Address: 0    Number: 0    Size: 0
ctime: 0x3f6765b5 -- Tue Sep 16 12:34:13 2003
atime: 0x3f676577 -- Tue Sep 16 12:33:11 2003
mtime: 0x3f65d2cb -- Mon Sep 15 07:55:07 2003
BLOCKS:
(0-11):8736-8747, (IND):8748, (12-1035):8749-9772, (DIND):9773, (IND):9774,
(103
6-2059):9775-10798, (IND):10799, (2060-3083):10800-11823, .....

 So why the debugfs reports a "Blockcount" number for (single files )
bigger
than the underlying file system's "Block count"? Does that means
the first
using a blocksize of 512 bytes, while the latter using 4096 bytes? 

 If so, that can explain why "df" works correctly since the size of
ando103.idx.2 is then 60126920*512=30784983040 bytes. same as "df"
reported
above. But from the report of debugfs, it has as well a wrong size reported:
"Size: 193750787024", where it comes from? and why?

 The problem is caused by two scp processes running at the same time, one
copying file ando103.idx.2, another copying file ando104.idx.2, to under /0.

Thanks.
--Guolin Cheng

--On 18 September 2003 00:35 -0700 Guolin Cheng <guolin@alexa.com> wrote:
> There are two big files above, and the total size is bigger then
> underlying file system size!! What a shame!
Sparse files?

Alex

On Thu, Sep 18, 2003 at 12:35:54AM -0700, Guolin Cheng
wrote:
> There are two big files above, and the total size is bigger then underlying
> file system size!! What a shame!
Don't Try This At Home Kid:

[clay@test1394 clay]$ dd if=/dev/zero of=BigBug bs=1M count=1
seek=1048575
1+0 records in
1+0 records out
[clay@test1394 clay]$ ls -lh BigBug
-rw-rw-r--    1 clay     clay         1.0T Sep 18 18:04 BigBug

Also, don't try this on a reiser(not sure about sparse file performance
on reiser4 ).

What a shame! SCNR ;)

-- 

Isaac Claymore                  /"\    ASCII Ribbon Campaign
Exavio Inc.                     \ /    Respect for open standards
Beijing, China                   X     No HTML/RTF in email
http://www.exavio.com.cn        / \    No M$ Word docs in email