I am trying to get a 9550SX to support a 1.5TB raid partition. I am unsure whether this is a driver problem, or an ext3 problem (as am getting some other wierdness detecting LUNs), but... fdisk recognizes the disk OK. I make a single extended partition with a single 1.5TB logical partition inside it. I then run mke2fs -j /dev/sdb It gets to writing inode tables, and wants to write 11176 block groups. It gets to 3600 (odd) of 11176 block groups fine, then slows to a complete crawl (2 secs per write), though strace shows it is fine. My suspicious mind says this is 1/3 of the way through the disk, which is 2^31 blocks, i.e. where the block number goes negative in a signed environment. Is there a problem with ext3 supporting more than 2^31 blocks? System: AMD64 2 x Opteron 275 Ubuntu Breezy mke2fs 1.38 Kernel 2.6.12 + 3w9xxx from 2.6.14 or from 3-ware 3-ware 9550SX RAID controller, latest firmware, BIOS RAID-5 1.5 TB single partition disk Alex
> I am trying to get a 9550SX to support a 1.5TB raid partition. I am unsure > whether this is a driver problem, or an ext3 problem (as am getting some > other wierdness detecting LUNs), but...I forgot to say, I think the underlying disk is OK and is not aliasing. I tried Stephen Tweedie's verify-data and it seems to be OK (I am taking it the error can, as it claims, be ignored). The same happens if I run it to /dev/sdb5. Furthermore, mkfs -treiserfs seems to run OK (though it takes a couple of minutes). So I am beginning to think this is ext3 (or ext3 tools) related. Alex amb at polonius:~/verify-data/verify-data-0.7$ sudo ./verify-data /dev/sdb 8G File is a block device: size 1499968045056 bytes (1.36T) Writing for 1.00M bytes every 8.00G bytes writing block 175 [offset 1.37T]...... Error writing file at offset 1503238553600 (1.37T): No space left on device (EOF, ignored) Completed writing file with 0 errors. Rereading for 1.00M bytes every 8.00G bytes reading block 174 [offset 1.36T]...... Completed reading file with 0 errors. amb at polonius:~/verify-data/verify-data-0.7$ sudo ./verify-data -S /dev/sdb 8G File is a block device: size 1499968045056 bytes (1.36T) Writing for 1.00M bytes every 8.00G bytes writing block 175 [offset 1.37T]...... Error writing file at offset 1503238553600 (1.37T): No space left on device (EOF, ignored) Completed writing file with 0 errors. Rereading for 1.00M bytes every 8.00G bytes reading block 174 [offset 1.36T]...... Completed reading file with 0 errors. amb at polonius:~/verify-data/verify-data-0.7$ sudo ./verify-data -F /dev/sdb 8G File is a block device: size 1499968045056 bytes (1.36T) Writing for 1.00M bytes every 8.00G bytes writing block 175 [offset 1.37T]...... Error writing file at offset 1503238553600 (1.37T): No space left on device (EOF, ignored) Completed writing file with 0 errors. Rereading for 1.00M bytes every 8.00G bytes reading block 174 [offset 1.36T]...... Completed reading file with 0 errors.
Andreas Dilger
2005-Oct-31 22:02 UTC
1.5TB ext3 partitions - mke2fs problems at 2^31 blocks
On Oct 31, 2005 09:01 +0000, Alex Bligh wrote:> I am trying to get a 9550SX to support a 1.5TB raid partition. I am unsure > whether this is a driver problem, or an ext3 problem (as am getting some > other wierdness detecting LUNs), but... > > fdisk recognizes the disk OK. I make a single extended partition with a > single 1.5TB logical partition inside it. I then run > > mke2fs -j /dev/sdb > > It gets to writing inode tables, and wants to write 11176 block groups. > It gets to 3600 (odd) of 11176 block groups fine, then slows to a complete > crawl (2 secs per write), though strace shows it is fine. > > My suspicious mind says this is 1/3 of the way through the disk, which is > 2^31 blocks, i.e. where the block number goes negative in a signed > environment. > > Is there a problem with ext3 supporting more than 2^31 blocks?We have customers using partitions close to 2TB, with 2.4.21 RHEL3, 2.6.5 SLES9, 2.6.9 RHEL4. It is concievable that your SCSI card/driver is not doing proper cache after 1TB or somehow starting to do strange things with the PCI bus after that point. Try doing something like: dd if=/dev/zero of=/dev/sdb bs=1M & sleep 1; while killall -USR1 dd; sleep 1; done and then plot the dd progress vs time to see if the rate is dramatically different over time. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
Damian Menscher
2005-Oct-31 22:10 UTC
1.5TB ext3 partitions - mke2fs problems at 2^31 blocks
On Mon, 31 Oct 2005, Alex Bligh wrote:> mke2fs -j /dev/sdb > > My suspicious mind says this is 1/3 of the way through the disk, which is > 2^31 blocks, i.e. where the block number goes negative in a signed > environment. > > Is there a problem with ext3 supporting more than 2^31 blocks? > > System: > AMD64 2 x Opteron 275 > Ubuntu Breezy > mke2fs 1.38 > Kernel 2.6.12 + 3w9xxx from 2.6.14 or from 3-ware > 3-ware 9550SX RAID controller, latest firmware, BIOS > RAID-5 1.5 TB single partition diskFWIW, we've been running a 3.5T partition for about a year, on x86_64-smp FC4. We had to compile the latest parted to create the GPT partition table, but other than that it wasn't a big problem. I can say that we had serious problems getting a >2TB partition working with a 32-bit machine, or a 2.4 kernel. Those are the problems that Andreas just referenced in his message in the other thread. :) I don't remember for sure whether we're actually USING more than 2TB of the 3.5 right now (and the machine is down today so I can't check). Damian Menscher -- -=#| <menscher at uiuc.edu> www.uiuc.edu/~menscher/ Ofc:(650)273-2757 |#=- -=#| The above opinions are not necessarily those of my employers. |#=-