Is the 8TB lun size limit going to be lifted any time soon? Because of this limit I''m only getting 33TB out of 45TB raw using raid5 with terabyte drives. What is everybody else doing for large arrays over 8TB? -Aaron
> Is the 8TB lun size limit going to be lifted any time soon? Because of > this limit I''m only getting 33TB out of 45TB raw using raid5 with > terabyte drives. What is everybody else doing for large arrays over 8TB? > > -AaronWe also have large >8TB disks we end up partitioning the disks into 8TB partitions and then adding them as OST''s to the filesystem. - David Brown
How is performance with two partitions on one array? On Nov 26, 2007, at 11:19 AM, David Brown wrote:>> Is the 8TB lun size limit going to be lifted any time soon? Because >> of >> this limit I''m only getting 33TB out of 45TB raw using raid5 with >> terabyte drives. What is everybody else doing for large arrays over >> 8TB? >> >> -Aaron > > We also have large >8TB disks we end up partitioning the disks into > 8TB partitions and then adding them as OST''s to the filesystem. > > - David BrownAaron Knister Associate Systems Administrator/Web Designer Center for Research on Environment and Water (301) 595-7001 aaron at iges.org
> How is performance with two partitions on one array?It doesn''t go as fast as it should but the system is an archive type and isn''t really meant for performance. - David Brown
Dividing them into raid5 stripes of ever fewer disks..... 8 TB is an ext3 limit. Other filesystems such as xfs have a much higher limit. Joe Aaron Knister wrote:> Is the 8TB lun size limit going to be lifted any time soon? Because of > this limit I''m only getting 33TB out of 45TB raw using raid5 with > terabyte drives. What is everybody else doing for large arrays over 8TB? > > -Aaron > > >
On Nov 26, 2007 11:08 -0500, Aaron Knister wrote:> Is the 8TB lun size limit going to be lifted any time soon? Because of > this limit I''m only getting 33TB out of 45TB raw using raid5 with > terabyte drives. What is everybody else doing for large arrays over 8TB?There will be no 8TB limit for new ZFS-based filesystems, available when Lustre 1.8 ships (planned for July ''08). For ext3 filesystems, the 8TB limit will be removed once the upstream kernel also removes this limit. For RHEL5 I believe they are working on up to 16TB per ext3 filesystem, but it isn''t done yet, nor have we tested it. For ext4 based file system code (available from about Linux 2.6.22 and on) the 8TB limit was also removed, but we need to port our ldiskfs patches over to be ext4 based. Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.
How difficult would it be to migrate from the ext3 based ldiskfs to zfs? On Nov 26, 2007, at 1:18 PM, Andreas Dilger wrote:> On Nov 26, 2007 11:08 -0500, Aaron Knister wrote: >> Is the 8TB lun size limit going to be lifted any time soon? Because >> of >> this limit I''m only getting 33TB out of 45TB raw using raid5 with >> terabyte drives. What is everybody else doing for large arrays over >> 8TB? > > There will be no 8TB limit for new ZFS-based filesystems, available > when > Lustre 1.8 ships (planned for July ''08). > > For ext3 filesystems, the 8TB limit will be removed once the upstream > kernel also removes this limit. For RHEL5 I believe they are working > on up to 16TB per ext3 filesystem, but it isn''t done yet, nor have we > tested it. For ext4 based file system code (available from about > Linux > 2.6.22 and on) the 8TB limit was also removed, but we need to port our > ldiskfs patches over to be ext4 based. > > Cheers, Andreas > -- > Andreas Dilger > Sr. Staff Engineer, Lustre Group > Sun Microsystems of Canada, Inc. >Aaron Knister Associate Systems Administrator/Web Designer Center for Research on Environment and Water (301) 595-7001 aaron at iges.org
On Nov 26, 2007 14:12 -0500, Aaron Knister wrote:> How difficult would it be to migrate from the ext3 based ldiskfs to zfs?It is planned that there will be an online object migrator. This will be useful for ext3->ZFS OST migration, and also space rebalancing or performance tuning of objects locations if there are different types of OSTs.> On Nov 26, 2007, at 1:18 PM, Andreas Dilger wrote: >> On Nov 26, 2007 11:08 -0500, Aaron Knister wrote: >>> Is the 8TB lun size limit going to be lifted any time soon? Because of >>> this limit I''m only getting 33TB out of 45TB raw using raid5 with >>> terabyte drives. What is everybody else doing for large arrays over 8TB? >> >> There will be no 8TB limit for new ZFS-based filesystems, available when >> Lustre 1.8 ships (planned for July ''08). >> >> For ext3 filesystems, the 8TB limit will be removed once the upstream >> kernel also removes this limit. For RHEL5 I believe they are working >> on up to 16TB per ext3 filesystem, but it isn''t done yet, nor have we >> tested it. For ext4 based file system code (available from about Linux >> 2.6.22 and on) the 8TB limit was also removed, but we need to port our >> ldiskfs patches over to be ext4 based. >> >> Cheers, Andreas >> -- >> Andreas Dilger >> Sr. Staff Engineer, Lustre Group >> Sun Microsystems of Canada, Inc. >> > > Aaron Knister > Associate Systems Administrator/Web Designer > Center for Research on Environment and Water > > (301) 595-7001 > aaron at iges.org > >Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.