The FAQ says that Lustre supports software RAID. Is the software RAID cluster aware? Can I put two partitions from two different nodes into the same RAID set? lin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20061205/27d643c8/attachment.html
On Dec 05, 2006 16:58 -0800, Lin Shen (lshen) wrote:> The FAQ says that Lustre supports software RAID. Is the software RAID > cluster aware? Can I put two partitions from two different nodes into > the same RAID set?No. The cluster-aware RAID feature (Lustre RAID = LAID) is only in the planning stages. This is only indicating that Lustre can use MD RAID{0,1,5} devices for a back-end OST or MDT filesystem. That said, there was a bug just found with Lustre OSTs and software RAID 5 (bug 11313) if you are planning on using software RAID 5 you need a patch. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
Hello Andreas, everyone, Im planning to use software RAID6 (mdadm) on an OST with Lustre. Do you think it is safe to do so ? Any advise against it ? Julien On 12/6/06, Andreas Dilger <adilger@clusterfs.com> wrote:> > On Dec 05, 2006 16:58 -0800, Lin Shen (lshen) wrote: > > The FAQ says that Lustre supports software RAID. Is the software RAID > > cluster aware? Can I put two partitions from two different nodes into > > the same RAID set? > > No. The cluster-aware RAID feature (Lustre RAID = LAID) is only in the > planning stages. > > This is only indicating that Lustre can use MD RAID{0,1,5} devices > for a back-end OST or MDT filesystem. That said, there was a bug just > found with Lustre OSTs and software RAID 5 (bug 11313) if you are > planning on using software RAID 5 you need a patch. > > Cheers, Andreas > -- > Andreas Dilger > Principal Software Engineer > Cluster File Systems, Inc. > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@clusterfs.com > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20061211/6557ff23/attachment.html
The versions of the Linux kernel for which there are Lustre patches do not have stable RAID6. Some of our customers experienced data corruption in the RAID6 layer. We have a highly optimized RAID5 layer for use with Lustre (several times faster than normal RAID5) - with that layer the SUN thumper boxes can get over 1GB/sec. Alex can point you at the kernel patches for that, if you''d like to try it. - Peter - On Dec 11, 2006, at 9:07 AM, Julien Perez wrote:> Hello Andreas, everyone, > > Im planning to use software RAID6 (mdadm) on an OST with Lustre. Do > you think it is safe to do so ? Any advise against it ? > > > Julien > > > On 12/6/06, Andreas Dilger <adilger@clusterfs.com> wrote: > On Dec 05, 2006 16:58 -0800, Lin Shen (lshen) wrote: > > The FAQ says that Lustre supports software RAID. Is the software > RAID > > cluster aware? Can I put two partitions from two different nodes > into > > the same RAID set? > > No. The cluster-aware RAID feature (Lustre RAID = LAID) is only in > the > planning stages. > > This is only indicating that Lustre can use MD RAID{0,1,5} devices > for a back-end OST or MDT filesystem. That said, there was a bug just > found with Lustre OSTs and software RAID 5 (bug 11313) if you are > planning on using software RAID 5 you need a patch. > > Cheers, Andreas > -- > Andreas Dilger > Principal Software Engineer > Cluster File Systems, Inc. > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@clusterfs.com > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@clusterfs.com > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss-------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20061211/8d15af64/attachment.html
Could you post a pointer to these patches to the list, I''d Like to look
at them as well.
Evan
________________________________
From: lustre-discuss-bounces@clusterfs.com
[mailto:lustre-discuss-bounces@clusterfs.com] On Behalf Of Peter J Braam
Sent: Monday, December 11, 2006 8:19 PM
To: Julien Perez
Cc: lustre-discuss@clusterfs.com; Andreas Dilger
Subject: Re: [Lustre-discuss] Software RAID support
The versions of the Linux kernel for which there are Lustre
patches do not have stable RAID6. Some of our customers experienced
data corruption in the RAID6 layer.
We have a highly optimized RAID5 layer for use with Lustre
(several times faster than normal RAID5) - with that layer the SUN
thumper boxes can get over 1GB/sec. Alex can point you at the kernel
patches for that, if you''d like to try it.
- Peter -
On Dec 11, 2006, at 9:07 AM, Julien Perez wrote:
Hello Andreas, everyone,
Im planning to use software RAID6 (mdadm) on an OST with
Lustre. Do you think it is safe to do so ? Any advise against it ?
Julien
On 12/6/06, Andreas Dilger <adilger@clusterfs.com>
wrote:
On Dec 05, 2006 16:58 -0800, Lin Shen (lshen)
wrote:
> The FAQ says that Lustre supports software
RAID. Is the software RAID
> cluster aware? Can I put two partitions from
two different nodes into
> the same RAID set?
No. The cluster-aware RAID feature (Lustre RAID
= LAID) is only in the
planning stages.
This is only indicating that Lustre can use MD
RAID{0,1,5} devices
for a back-end OST or MDT filesystem. That
said, there was a bug just
found with Lustre OSTs and software RAID 5 (bug
11313) if you are
planning on using software RAID 5 you need a
patch.
Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.
_______________________________________________
Lustre-discuss mailing list
Lustre-discuss@clusterfs.com
https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
_______________________________________________
Lustre-discuss mailing list
Lustre-discuss@clusterfs.com
https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20061212/5722b1bb/attachment.html
Hello Peter, Enquiring minds want to know: if this code does on the fly parity checking on reads, what protocol is being used between the OSS node and the thumper and whether the data rates quoted are to a single thumper or a group of thumpers behind the OSS node. I second the request by Evan for patch pointers. thanks, Jim On Mon, 11 Dec 2006, Peter J Braam wrote:> The versions of the Linux kernel for which there are Lustre patches do not > have stable RAID6. Some of our customers experienced data corruption in the > RAID6 layer. > > We have a highly optimized RAID5 layer for use with Lustre (several times > faster than normal RAID5) - with that layer the SUN thumper boxes can get > over 1GB/sec. Alex can point you at the kernel patches for that, if you''d > like to try it. > > - Peter - > > > On Dec 11, 2006, at 9:07 AM, Julien Perez wrote: > >> Hello Andreas, everyone, >> >> Im planning to use software RAID6 (mdadm) on an OST with Lustre. Do you >> think it is safe to do so ? Any advise against it ? >> >> >> Julien >> >> >> On 12/6/06, Andreas Dilger <adilger@clusterfs.com> wrote: >> On Dec 05, 2006 16:58 -0800, Lin Shen (lshen) wrote: >> > The FAQ says that Lustre supports software RAID. Is the software RAID >> > cluster aware? Can I put two partitions from two different nodes into >> > the same RAID set? >> >> No. The cluster-aware RAID feature (Lustre RAID = LAID) is only in the >> planning stages. >> >> This is only indicating that Lustre can use MD RAID{0,1,5} devices >> for a back-end OST or MDT filesystem. That said, there was a bug just >> found with Lustre OSTs and software RAID 5 (bug 11313) if you are >> planning on using software RAID 5 you need a patch. >> >> Cheers, Andreas >> -- >> Andreas Dilger >> Principal Software Engineer >> Cluster File Systems, Inc. >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss@clusterfs.com >> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss >> >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss@clusterfs.com >> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss > >
>>>>> Jim Garlick (JG) writes:JG> Hello Peter, JG> Enquiring minds want to know: if this code does on the fly parity JG> checking on reads, what protocol is being used between the OSS node JG> and the thumper and whether the data rates quoted are to a single JG> thumper or a group of thumpers behind the OSS node. the current code doesn''t check parity on reads. JG> I second the request by Evan for patch pointers. they are in our b1_4 and b1_5 branches. thanks, Alex
Hi Jim This is the number we have seen with 1 thumper. Alex - could you explain what patch series should be used? - Peter - On Dec 12, 2006, at 11:41 AM, Jim Garlick wrote:> Hello Peter, > > Enquiring minds want to know: if this code does on the fly parity > checking on reads, what protocol is being used between the OSS node > and the thumper and whether the data rates quoted are to a single > thumper or a group of thumpers behind the OSS node. > > I second the request by Evan for patch pointers. > > thanks, > > Jim > > On Mon, 11 Dec 2006, Peter J Braam wrote: > >> The versions of the Linux kernel for which there are Lustre >> patches do not have stable RAID6. Some of our customers >> experienced data corruption in the RAID6 layer. >> >> We have a highly optimized RAID5 layer for use with Lustre >> (several times faster than normal RAID5) - with that layer the SUN >> thumper boxes can get over 1GB/sec. Alex can point you at the >> kernel patches for that, if you''d like to try it. >> >> - Peter - >> >> >> On Dec 11, 2006, at 9:07 AM, Julien Perez wrote: >> >>> Hello Andreas, everyone, >>> Im planning to use software RAID6 (mdadm) on an OST with Lustre. >>> Do you think it is safe to do so ? Any advise against it ? >>> Julien >>> On 12/6/06, Andreas Dilger <adilger@clusterfs.com> wrote: >>> On Dec 05, 2006 16:58 -0800, Lin Shen (lshen) wrote: >>> > The FAQ says that Lustre supports software RAID. Is the >>> software RAID >>> > cluster aware? Can I put two partitions from two different >>> nodes into >>> > the same RAID set? >>> No. The cluster-aware RAID feature (Lustre RAID = LAID) is only >>> in the >>> planning stages. >>> This is only indicating that Lustre can use MD RAID{0,1,5} devices >>> for a back-end OST or MDT filesystem. That said, there was a bug >>> just >>> found with Lustre OSTs and software RAID 5 (bug 11313) if you are >>> planning on using software RAID 5 you need a patch. >>> Cheers, Andreas >>> -- >>> Andreas Dilger >>> Principal Software Engineer >>> Cluster File Systems, Inc. >>> _______________________________________________ >>> Lustre-discuss mailing list >>> Lustre-discuss@clusterfs.com >>> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss >>> _______________________________________________ >>> Lustre-discuss mailing list >>> Lustre-discuss@clusterfs.com >>> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss >> >>
2.6-rhel4.series contains all the patches except zero-copy where the bug was fixed a week ago. 2.6-rhel4-titech.series contains all the patches. thanks, Alex>>>>> Peter J Braam (PJB) writes:PJB> Hi Jim PJB> This is the number we have seen with 1 thumper. PJB> Alex - could you explain what patch series should be used?
On Tuesday 12 December 2006 19:46, Alex Tomas wrote:> >>>>> Jim Garlick (JG) writes: > > JG> Hello Peter, > JG> Enquiring minds want to know: if this code does on the fly parity > JG> checking on reads, what protocol is being used between the OSS node > JG> and the thumper and whether the data rates quoted are to a single > JG> thumper or a group of thumpers behind the OSS node. > > the current code doesn''t check parity on reads. > > JG> I second the request by Evan for patch pointers. > > they are in our b1_4 and b1_5 branches.And where can these branches be accessed, please? Regards, Erich
On Dec 13, 2006 11:46 +0100, Erich Focht wrote:> > >>>>> Jim Garlick (JG) writes: > > JG> Enquiring minds want to know: if this code does on the fly parity > > JG> checking on reads, what protocol is being used between the OSS node > > JG> and the thumper and whether the data rates quoted are to a single > > JG> thumper or a group of thumpers behind the OSS node. > > > > they are in our b1_4 and b1_5 branches. > > And where can these branches be accessed, please?The branches are the 1.4.x and 1.5.x release source, so the raid5-* patches are included in every Lustre release. Please have a look in lustre/kernel_patches/patches in the sources. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
On Wednesday 13 December 2006 22:25, Andreas Dilger wrote:> On Dec 13, 2006 11:46 +0100, Erich Focht wrote: > > > >>>>> Jim Garlick (JG) writes: > > > JG> Enquiring minds want to know: if this code does on the fly parity > > > JG> checking on reads, what protocol is being used between the OSS node > > > JG> and the thumper and whether the data rates quoted are to a single > > > JG> thumper or a group of thumpers behind the OSS node. > > > > > > they are in our b1_4 and b1_5 branches. > > > > And where can these branches be accessed, please? > > The branches are the 1.4.x and 1.5.x release source, so the raid5-* > patches are included in every Lustre release. Please have a look in > lustre/kernel_patches/patches in the sources.I thought Alex mentioned there were fixes from 1 week ago, but maybe he didn''t mean the raid5 patches. In 1.5.95 I see: raid5-configurable-cachesize.patch raid5-large-io.patch raid5-merge-ios.patch raid5-optimize-memcpy.patch raid5-serialize-ovelapping-reqs.patch raid5-stats.patch raid5-stripe-by-stripe-handling.patch Any major bug besides these? Thanks, best regards, Erich
On Dec 14, 2006 00:09 +0100, Erich Focht wrote:> I thought Alex mentioned there were fixes from 1 week ago, but maybe he didn''t > mean the raid5 patches. In 1.5.95 I see: > > raid5-configurable-cachesize.patch > raid5-large-io.patch > raid5-merge-ios.patch > raid5-optimize-memcpy.patch > raid5-serialize-ovelapping-reqs.patch > raid5-stats.patch > raid5-stripe-by-stripe-handling.patch > > Any major bug besides these?Yes, there is one bug that was just fixed in RAID5 and isn''t in a release yet. There is an open bug 11313 with a patch. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
Will LAID be available by end of 2007? If not, how hard is it to add only RAID 0 and 1 across nodes? Thanks lin> -----Original Message----- > From: Andreas Dilger [mailto:adilger@clusterfs.com] > Sent: Tuesday, December 05, 2006 11:41 PM > To: Lin Shen (lshen) > Cc: lustre-discuss@clusterfs.com > Subject: Re: [Lustre-discuss] Software RAID support > > On Dec 05, 2006 16:58 -0800, Lin Shen (lshen) wrote: > > The FAQ says that Lustre supports software RAID. Is the > software RAID > > cluster aware? Can I put two partitions from two different > nodes into > > the same RAID set? > > No. The cluster-aware RAID feature (Lustre RAID = LAID) is > only in the planning stages. > > This is only indicating that Lustre can use MD RAID{0,1,5} > devices for a back-end OST or MDT filesystem. That said, > there was a bug just found with Lustre OSTs and software RAID > 5 (bug 11313) if you are planning on using software RAID 5 > you need a patch. > > Cheers, Andreas > -- > Andreas Dilger > Principal Software Engineer > Cluster File Systems, Inc. >
Hi, Lustre network RAID0 is available now (and has been for years). Network RAID1 is essentially equally difficult as the general problem, we do not know if it will be done late 2007, but it is under development. - Peter - Lin Shen (lshen) wrote:> Will LAID be available by end of 2007? If not, how hard is it to add > only RAID 0 and 1 across nodes? > > Thanks > lin > > >> -----Original Message----- >> From: Andreas Dilger [mailto:adilger@clusterfs.com] >> Sent: Tuesday, December 05, 2006 11:41 PM >> To: Lin Shen (lshen) >> Cc: lustre-discuss@clusterfs.com >> Subject: Re: [Lustre-discuss] Software RAID support >> >> On Dec 05, 2006 16:58 -0800, Lin Shen (lshen) wrote: >> >>> The FAQ says that Lustre supports software RAID. Is the >>> >> software RAID >> >>> cluster aware? Can I put two partitions from two different >>> >> nodes into >> >>> the same RAID set? >>> >> No. The cluster-aware RAID feature (Lustre RAID = LAID) is >> only in the planning stages. >> >> This is only indicating that Lustre can use MD RAID{0,1,5} >> devices for a back-end OST or MDT filesystem. That said, >> there was a bug just found with Lustre OSTs and software RAID >> 5 (bug 11313) if you are planning on using software RAID 5 >> you need a patch. >> >> Cheers, Andreas >> -- >> Andreas Dilger >> Principal Software Engineer >> Cluster File Systems, Inc. >> >> > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@clusterfs.com > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20061221/b35415b2/attachment.html