i'm trying to set up a hardware RAID array (eRAID from DatOptic) under CentOS. we have two more of these arrays which have worked very well for their purposes (backup and/or portable storage), but neither of our older arrays exceeded the dreaded 2TB barrier (one of them is, in fact, 2TB). the server i'm currently testing with is running CentOS4.5, but i do have CentOS5 servers i could use. the problem i'm running into is that the array SHOULD be recognized as ~3TB, but i'm unable (even using "parted") to see it as more than ~2TB. it sounds suspiciously like the typical "2TB limitation" problem, but i was under the impression that CentOS4 (and 5) can handle much larger (8TB?) filesystems. in fact, the kernel "USB Mass Storage" driver that detects the array when i plug it in only detects the device as 2TB as well. (so, it could easily be a limitation of that hardware driver, although i couldn't find any specification for that limitation when i googled for it). i should say, although i dread the possible suggestions that may arise from admitting it, that the array also supports FireWire and SATA connections. (we are STRONGLY inclined to wanting to use the array as USB, as our primary concern is easy portability and not speed) the RAID array is the DatOptic eRAID, with 5 750GB disks in a RAID-5 array. the array itself seemed to have no problem creating the ~3TB volume, but it does not detect as that size when i hook it up. here are the relevant kernel logs: SCSI subsystem initialized Initializing USB Mass Storage driver... scsi0 : SCSI emulation for USB Mass Storage devices Vendor: USB 2.0 Model: Storage Device Rev: 0111 Type: Direct-Access ANSI SCSI revision: 02 USB Mass Storage device found at 3 usbcore: registered new driver usb-storage USB Mass Storage support registered. SCSI device sda: 4294920192 512-byte hdwr sectors (2198999 MB) sda: assuming drive cache: write through SCSI device sda: 4294920191 512-byte hdwr sectors (2198999 MB) sda: assuming drive cache: write through sda: unknown partition table Attached scsi disk sda at scsi0, channel 0, id 0, lun 0 and here's what "parted" also says about the array: Disk geometry for /dev/sda: 0.000-2097128.999 megabytes Disk label type: gpt Minor Start End Filesystem Name Flags it's important that we get full use of the array (1TB is a lot of space to waste). anyone have any ideas, suggestions, comments, or criticisms? any help would be greatly appreciated... B. Karhan simon at pop.psu.edu PRI/SSRI Unix Administrator
On 6/7/07, Benjamin Karhan <simon at pop.psu.edu> wrote:> > i'm trying to set up a hardware RAID array (eRAID from DatOptic) > under CentOS. we have two more of these arrays which have worked > very well for their purposes (backup and/or portable storage), but > neither of our older arrays exceeded the dreaded 2TB barrier > (one of them is, in fact, 2TB).Some raid devices themselves have a 2TB limit, and you either have to enable spanning on the raid card, or use the OS to work around this.> the RAID array is the DatOptic eRAID, with 5 750GB disks in a > RAID-5 array. the array itself seemed to have no problem > creating the ~3TB volume, but it does not detect as that > size when i hook it up.You used parted and set the partition type as GPT right? If it's not GPT, then you'll hit the 2TB limit with the filesystem.> Disk geometry for /dev/sda: 0.000-2097128.999 megabytes > Disk label type: gpt > Minor Start End Filesystem Name FlagsYep, seems you're using GPT. What filesystem are you using for this? I didn't see that informaiton in the provided info, unless I overlooked it.> it's important that we get full use of the array (1TB is a lot of > space to waste). > anyone have any ideas, suggestions, comments, or criticisms? > any help would be greatly appreciated...Very much so. Worst case I'd say split it up into 1TB chunks and use LVM to join them all. -- During times of universal deceit, telling the truth becomes a revolutionary act. George Orwell
On Thu, 7 Jun 2007 at 8:19pm, Benjamin Karhan wrote> in fact, the kernel "USB Mass Storage" driver that detects the > array when i plug it in only detects the device as 2TB as well. > (so, it could easily be a limitation of that hardware driver, > although i couldn't find any specification for that limitation > when i googled for it).My googling seemed to indicate that, yes, the USB mass storage protocol (not just the Linux driver, but the protocol itself) is limited to 2TB per LUN.> i should say, although i dread the possible suggestions that may > arise from admitting it, that the array also supports FireWire > and SATA connections. (we are STRONGLY inclined to wanting to > use the array as USB, as our primary concern is easy portability > and not speed)I can't see anything that says Firewire has the same 2TB limitation as USB. So that's worth a shot.> the RAID array is the DatOptic eRAID, with 5 750GB disks in a > RAID-5 array. the array itself seemed to have no problem > creating the ~3TB volume, but it does not detect as that > size when i hook it up.Many such devices have an "auto-carve" feature (or some such), that automatically splits one RAID volume into <2TB chunks, each on a different LUN. If you're married to USB, see if your hardware will do that. -- Joshua Baker-LePain Department of Biomedical Engineering Duke University