I know some users talked about DRBD for the shared disk on the MDS. What was the conclusion of this? Bad Idea? I do some high available NFS using this exact same setup. DRBD provides shared storage, Heart Beat is used to monitor hosts. IPMI is used by HeartBeat to power down hosts that are to be killed. The plan on our table right now is two thumpers as the OSS''s. Then two x4100 or 4200/s with mirrors SAS drives then shared across with DRBD with Heart Beat. Any comments? Any issues to be aware of? Anyone running something similar? Brock Palen www.umich.edu/~brockp Center for Advanced Computing brockp at umich.edu (734)936-1985
On Tue, 2008-05-06 at 17:00 -0400, Brock Palen wrote:> I know some users talked about DRBD for the shared disk on the MDS. > > What was the conclusion of this? Bad Idea? I do some high available > NFS using this exact same setup.You might want to follow along on bug 15710. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080506/1789d557/attachment.bin
Hi, we are testing a similar setup, a HA-pair of servers for MGS/MDS, were the HA is provided by DRBD and Heartbeat. So far we did not observe any problems. However, in the current ''production'' mode of our cluster, there hasn''t been a HA-failover, so nothing could have gone wrong with that part of the setup. In the test phase before, for instance once the power cables of the primary server were ripped out while someone was writing with iozone to the filesystem: nothing happened. These were large-file-writes, however. Writing, say, 5MB files, you certainly would notice the short disappearance of the MDT as a short interruption. But no problems with inconsistencies between the two DRBD-MDT-partitions. Now, if you plan to do that on the OSSs, I have no experience with that, but there is bug 15710, as mentioned by Brian, which is also positive about the use or DRBD. Regards, Thomas Brock Palen wrote:> I know some users talked about DRBD for the shared disk on the MDS. > > What was the conclusion of this? Bad Idea? I do some high available > NFS using this exact same setup. > > > DRBD provides shared storage, > Heart Beat is used to monitor hosts. > IPMI is used by HeartBeat to power down hosts that are to be killed. > > The plan on our table right now is two thumpers as the OSS''s. > Then two x4100 or 4200/s with mirrors SAS drives then shared across > with DRBD with Heart Beat. > > > Any comments? Any issues to be aware of? Anyone running something > similar? > > > Brock Palen > www.umich.edu/~brockp > Center for Advanced Computing > brockp at umich.edu > (734)936-1985 > > > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss-- -------------------------------------------------------------------- Thomas Roth Department: Informationstechnologie Location: SB3 1.262 Phone: +49-6159-71 1453 Fax: +49-6159-71 2986 Gesellschaft f?r Schwerionenforschung mbH Planckstra?e 1 D-64291 Darmstadt www.gsi.de Gesellschaft mit beschr?nkter Haftung Sitz der Gesellschaft: Darmstadt Handelsregister: Amtsgericht Darmstadt, HRB 1528 Gesch?ftsf?hrer: Professor Dr. Horst St?cker Vorsitzende des Aufsichtsrates: Dr. Beatrix Vierkorn-Rudolph, Stellvertreter: Ministerialdirigent Dr. Rolf Bernhardt
On May 7, 2008, at 4:55 AM, Thomas Roth wrote:> Hi, > > we are testing a similar setup, a HA-pair of servers for MGS/MDS, > were the HA is provided by DRBD and Heartbeat. > So far we did not observe any problems. However, in the current > ''production'' mode of our cluster, there hasn''t been a HA-failover, > so nothing could have gone wrong with that part of the setup. > In the test phase before, for instance once the power cables of the > primary server were ripped out while someone was writing with > iozone to the filesystem: nothing happened. These were large-file- > writes, however. Writing, say, 5MB files, you certainly would > notice the short disappearance of the MDT as a short interruption. > But no problems with inconsistencies between the two DRBD-MDT- > partitions.This is good to know. But we are not sure we want to risk that. I have used DRBD before. The default ''protocol'' for drbd is to block until both disk are written. What about an iSCSI cabnet? Sun (who we are buying the thumpers from for OST''s) has a very good academic price for a 2510 SAS array. Has anyone ever used one of these ''Sun StorageTek 2510'' arrays? Problems with iSCSI and MDS? Remember the goal is to build a fail over pair of MDS''s.> > Now, if you plan to do that on the OSSs, I have no experience with > that, but there is bug 15710, as mentioned by Brian, which is also > positive about the use or DRBD. > > Regards, > Thomas > > Brock Palen wrote: >> I know some users talked about DRBD for the shared disk on the MDS. >> What was the conclusion of this? Bad Idea? I do some high >> available NFS using this exact same setup. >> DRBD provides shared storage, >> Heart Beat is used to monitor hosts. >> IPMI is used by HeartBeat to power down hosts that are to be killed. >> The plan on our table right now is two thumpers as the OSS''s. >> Then two x4100 or 4200/s with mirrors SAS drives then shared >> across with DRBD with Heart Beat. >> Any comments? Any issues to be aware of? Anyone running >> something similar? >> Brock Palen >> www.umich.edu/~brockp >> Center for Advanced Computing >> brockp at umich.edu >> (734)936-1985 >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-discuss > > -- > -------------------------------------------------------------------- > Thomas Roth > Department: Informationstechnologie > Location: SB3 1.262 > Phone: +49-6159-71 1453 Fax: +49-6159-71 2986 > > Gesellschaft f?r Schwerionenforschung mbH > Planckstra?e 1 > D-64291 Darmstadt > www.gsi.de > > Gesellschaft mit beschr?nkter Haftung > Sitz der Gesellschaft: Darmstadt > Handelsregister: Amtsgericht Darmstadt, HRB 1528 > > Gesch?ftsf?hrer: Professor Dr. Horst St?cker > > Vorsitzende des Aufsichtsrates: Dr. Beatrix Vierkorn-Rudolph, > Stellvertreter: Ministerialdirigent Dr. Rolf Bernhardt > >