All, I am starting to try and configure failover for our lustre filesystem. Node00 is the mgs/mdt Node00 is the oss for ost0 and failnode for ost1 Node01 is the oss for ost1 and failnode for ost0 Both osts are on an SRP network and are visible by both nodes. Ost0 is mounted on node00 Ost1 is mounted on node01 If I try to mount ost0 on node01 I see in the logs for node00: kernel: Lustre: Denying initial registration attempt from nid 10.100.255.250 at o2ib, specified as failover So do I have to manually mount the ost for failover purposes when there is a fail? I would have thought I mount the osts on both nodes and lustre will manage which node is the active node. Brian Andrus
Brian, Do you have corosync or other Linux HA software infrastructure running on these systems? You need an HA software layer to manage heartbeat monitoring, split-brain protection and mounting/migrating of resources. --Jeff On 10/11/12 2:02 PM, Andrus, Brian Contractor wrote:> All, > > I am starting to try and configure failover for our lustre filesystem. > Node00 is the mgs/mdt > Node00 is the oss for ost0 and failnode for ost1 > Node01 is the oss for ost1 and failnode for ost0 > > Both osts are on an SRP network and are visible by both nodes. > Ost0 is mounted on node00 > Ost1 is mounted on node01 > > If I try to mount ost0 on node01 I see in the logs for node00: > kernel: Lustre: Denying initial registration attempt from nid 10.100.255.250 at o2ib, specified as failover > > So do I have to manually mount the ost for failover purposes when there is a fail? > I would have thought I mount the osts on both nodes and lustre will manage which node is the active node. > > > Brian Andrus > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss-- ------------------------------ Jeff Johnson Co-Founder Aeon Computing jeff.johnson at aeoncomputing.com www.aeoncomputing.com t: 858-412-3810 x101 f: 858-412-3845 m: 619-204-9061 /* New Address */ 4170 Morena Boulevard, Suite D - San Diego, CA 92117
You must manually mount them in only one place. When a failure occurs you have to manually mount the OST on the failover OSS (or have a piece of software do it). -----Original Message----- From: lustre-discuss-bounces at lists.lustre.org [mailto:lustre-discuss-bounces at lists.lustre.org] On Behalf Of Andrus, Brian Contractor Sent: Thursday, October 11, 2012 5:02 PM To: lustre-discuss at lists.lustre.org Subject: [Lustre-discuss] mounting Failover OSTs All, I am starting to try and configure failover for our lustre filesystem. Node00 is the mgs/mdt Node00 is the oss for ost0 and failnode for ost1 Node01 is the oss for ost1 and failnode for ost0 Both osts are on an SRP network and are visible by both nodes. Ost0 is mounted on node00 Ost1 is mounted on node01 If I try to mount ost0 on node01 I see in the logs for node00: kernel: Lustre: Denying initial registration attempt from nid 10.100.255.250 at o2ib, specified as failover So do I have to manually mount the ost for failover purposes when there is a fail? I would have thought I mount the osts on both nodes and lustre will manage which node is the active node. Brian Andrus _______________________________________________ Lustre-discuss mailing list Lustre-discuss at lists.lustre.org http://lists.lustre.org/mailman/listinfo/lustre-discuss
Hi Andrus, Lustre does not fully support failover. It only allows to configure OST so it can be accessed on more then one OSS. However handling a failure and operation of mounting device on the failover OSS is not automatically done by Lustre. You can not mount OST on both OSS servers simultaneously that would cause a corruption of the ldiskfs filesystem. Fortunately since some time ldiskfs comes with mmp (multi mount protection) flag which will stop you doing that (and this is probably what you are seeing in the logs). As Jeff says to fully automatically configure failover in lustre you will need to use additional HA software like Heartbeat with Pacemaker or Corosync with Pacemaker. There are others as well but these two are most popular I think. Best regards, Wojciech On 11 October 2012 22:02, Andrus, Brian Contractor <bdandrus at nps.edu> wrote:> All, > > I am starting to try and configure failover for our lustre filesystem. > Node00 is the mgs/mdt > Node00 is the oss for ost0 and failnode for ost1 > Node01 is the oss for ost1 and failnode for ost0 > > Both osts are on an SRP network and are visible by both nodes. > Ost0 is mounted on node00 > Ost1 is mounted on node01 > > If I try to mount ost0 on node01 I see in the logs for node00: > kernel: Lustre: Denying initial registration attempt from nid > 10.100.255.250 at o2ib, specified as failover > > So do I have to manually mount the ost for failover purposes when there is > a fail? > I would have thought I mount the osts on both nodes and lustre will manage > which node is the active node. > > > Brian Andrus > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-discuss >-- Wojciech Turek Senior System Architect High Performance Computing Service University of Cambridge -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-discuss/attachments/20121012/3081c742/attachment.html