Hi everyone,
This is a strange problem I have been having. I'm not sure where the
problem is, so I figured I'd start here.
I as having problems with Bacula stopping on 17Gig Volume sizes, so I
decided to try to Just dd a 50 gig file. Sure enough, once the file hit
17 gigs dd stopped and spit out an error
(pandora bacula)# dd if=/dev/zero of=bigfile bs=1M count=50000
File size limit exceeded
(pandora bacula)#
(pandora bacula)# ll
total 20334813
-rw-r--r-- 1 root root 17247252480 Sep 23 00:44 bigfile
-rw-r----- 1 root root 302323821 Sep 23 01:10 Default-0001
-rw-r----- 1 root root 156637059 Sep 18 01:08 Diff-wi0001
-rw-r----- 1 root root 46985831 Sep 6 19:38 Full-0001
-rw-r----- 1 root root 47126293 Sep 7 14:39 Full-0002
-rw-r----- 1 root root 2841621607 Sep 13 17:11 Full-wi0001
-rw-r----- 1 root root 1584252 Sep 18 01:05 Inc-0001
-rw-r----- 1 root root 97963834 Sep 14 01:05 Inc-wi0001
Filesystem Size Used Avail Use% Mounted on
/dev/hda2 9.7G 5.8G 3.4G 64% /
/dev/hda1 99M 20M 75M 21% /boot
/dev/hda4 102G 2.2G 94G 3% /home
/dev/md2 221G 90G 120G 43% /mnt/storage
none 1014M 0 1014M 0% /dev/shm
/dev/mapper/lvg01-coraid
812G 693G 114G 86% /mnt/coraid
There are a few layers on this partation, so I figured I'd start at the
top with you guys and work my way down. The partation this size limit
is on looks like so...
/mnt/coraid
+--------+
| Ext3 |
+--------+
| LVM 2 |
+--------+
| Raid 5 |
+--------+
So any one of these layers could be the problem. I was able to create a
100 Gig file on the /home partition, so perhaps ext3 is not the problem,
but I'm really not sure.
The system is CentOs 4.1 running 2.6.13.2 (also tried 2.6.12.2)
Any insight would be great
--
Thanks,
Brandon Evans
"I wouldn't recommend sex, drugs or insanity for everyone, but
they've
always worked for me."
-Hunter S. Thompson
What does "ulimit -a" report for your maximum allowed file size? Could you have limited yourself somehow? -- Matt Stegman On Fri, 23 Sep 2005, Brandon Evans wrote:> Hi everyone, > This is a strange problem I have been having. I'm not sure where the > problem is, so I figured I'd start here. > > I as having problems with Bacula stopping on 17Gig Volume sizes, so I > decided to try to Just dd a 50 gig file. Sure enough, once the file hit > 17 gigs dd stopped and spit out an error > > > (pandora bacula)# dd if=/dev/zero of=bigfile bs=1M count=50000 > File size limit exceeded > (pandora bacula)# > > > (pandora bacula)# ll > total 20334813 > -rw-r--r-- 1 root root 17247252480 Sep 23 00:44 bigfile > -rw-r----- 1 root root 302323821 Sep 23 01:10 Default-0001 > -rw-r----- 1 root root 156637059 Sep 18 01:08 Diff-wi0001 > -rw-r----- 1 root root 46985831 Sep 6 19:38 Full-0001 > -rw-r----- 1 root root 47126293 Sep 7 14:39 Full-0002 > -rw-r----- 1 root root 2841621607 Sep 13 17:11 Full-wi0001 > -rw-r----- 1 root root 1584252 Sep 18 01:05 Inc-0001 > -rw-r----- 1 root root 97963834 Sep 14 01:05 Inc-wi0001 > > Filesystem Size Used Avail Use% Mounted on > /dev/hda2 9.7G 5.8G 3.4G 64% / > /dev/hda1 99M 20M 75M 21% /boot > /dev/hda4 102G 2.2G 94G 3% /home > /dev/md2 221G 90G 120G 43% /mnt/storage > none 1014M 0 1014M 0% /dev/shm > /dev/mapper/lvg01-coraid > 812G 693G 114G 86% /mnt/coraid > > > > There are a few layers on this partation, so I figured I'd start at the > top with you guys and work my way down. The partation this size limit > is on looks like so... > > /mnt/coraid > +--------+ > | Ext3 | > +--------+ > | LVM 2 | > +--------+ > | Raid 5 | > +--------+ > > > So any one of these layers could be the problem. I was able to create a > 100 Gig file on the /home partition, so perhaps ext3 is not the problem, > but I'm really not sure. > > > The system is CentOs 4.1 running 2.6.13.2 (also tried 2.6.12.2) > > Any insight would be great > >
On Fri, Sep 23, 2005 at 02:43:29PM -0700, Brandon Evans wrote:> Hi everyone, > This is a strange problem I have been having. I'm not sure where the > problem is, so I figured I'd start here. > > I as having problems with Bacula stopping on 17Gig Volume sizes, so I > decided to try to Just dd a 50 gig file. Sure enough, once the file hit > 17 gigs dd stopped and spit out an error > > (pandora bacula)# dd if=/dev/zero of=bigfile bs=1M count=50000 > File size limit exceeded > (pandora bacula)# > > (pandora bacula)# ll > total 20334813 > -rw-r--r-- 1 root root 17247252480 Sep 23 00:44 bigfileIf you are using a 1k filesystem, then a file can consist of ten direct blocks, plus 256 data blocks addressed via the indirect block, plus 256*256 data blocks addressed from the indirect block, plus 256*256*256 data blocks from the triple-indirect block: (10 + 256 + 256*256 + 256*256*256) * 1024 = 17247252480 Does that number look familiar? So the problem is that you created the file system using a 1k blocksize. Filesystems with a 1k blocksize are horribly inefficient for large files, and they max out at a little over a little over 16 gigabytes. (Note that 16 gigs is 17179869184 bytes, unless you are a disk drive company in which case your marketing department calls it 17 gigs. :-) - Ted> -rw-r----- 1 root root 302323821 Sep 23 01:10 Default-0001 > -rw-r----- 1 root root 156637059 Sep 18 01:08 Diff-wi0001 > -rw-r----- 1 root root 46985831 Sep 6 19:38 Full-0001 > -rw-r----- 1 root root 47126293 Sep 7 14:39 Full-0002 > -rw-r----- 1 root root 2841621607 Sep 13 17:11 Full-wi0001 > -rw-r----- 1 root root 1584252 Sep 18 01:05 Inc-0001 > -rw-r----- 1 root root 97963834 Sep 14 01:05 Inc-wi0001 > > Filesystem Size Used Avail Use% Mounted on > /dev/hda2 9.7G 5.8G 3.4G 64% / > /dev/hda1 99M 20M 75M 21% /boot > /dev/hda4 102G 2.2G 94G 3% /home > /dev/md2 221G 90G 120G 43% /mnt/storage > none 1014M 0 1014M 0% /dev/shm > /dev/mapper/lvg01-coraid > 812G 693G 114G 86% /mnt/coraid > > > > There are a few layers on this partation, so I figured I'd start at the > top with you guys and work my way down. The partation this size limit > is on looks like so... > > /mnt/coraid > +--------+ > | Ext3 | > +--------+ > | LVM 2 | > +--------+ > | Raid 5 | > +--------+ > > > So any one of these layers could be the problem. I was able to create a > 100 Gig file on the /home partition, so perhaps ext3 is not the problem, > but I'm really not sure. > > > The system is CentOs 4.1 running 2.6.13.2 (also tried 2.6.12.2) > > Any insight would be great > > -- > > Thanks, > Brandon Evans > > "I wouldn't recommend sex, drugs or insanity for everyone, but they've > always worked for me." > -Hunter S. Thompson > > _______________________________________________ > Ext3-users mailing list > Ext3-users at redhat.com > https://www.redhat.com/mailman/listinfo/ext3-users